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Lecture 1 

Introduction and Overview 

What is Algebra? 

History 

Algebra is named in honor of Mohammed Ibn-e- Musa al-Khowarizmi. Around 825, he 
wrote a book entitled Hisb al-jabr u'l muqubalah, ("the science of reduction and 
cancellation"). His book, Al-jabr, presented rules for solving equations. 

Algebra is a branch of Mathematics that uses mathematical statements to describe 
relationships between things that vary over time. These variables include things like the 
relationship between supply of an object and its price. When we use a mathematical 
statement to describe a relationship, we often use letters to represent the quantity that 
varies, since it is not a fixed amount. These letters and symbols are referred to as 
variables. 

Algebra is a part of mathematics in which unknown quantities are found with the help of 
relations between the unknown and known. 

In algebra, letters are sometimes used in place of numbers. 

The mathematical statements that describe relationships are expressed using algebraic 
terms, expressions, or equations (mathematical statements containing letters or symbols 
to represent numbers). Before we use algebra to find infonnation about these kinds of 
relationships, it is important to first introduce some basic tenninology. 

Algebraic Term 

The basic unit of an algebraic expression is a term. In general, a tenn is either a product 
of a number and with one or more variables. 

For example 4x is an algebraic term in which 4 is coefficient and x is said to be variable. 

Study of Algebra 

Today, algebra is the study of the properties of operations on numbers. Algebra 
generalizes arithmetic by using symbols, usually letters, to represent numbers or 
unknown quantities. Algebra is a problem-solving tool. It is like a tractor, which is a 
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farmer's tool. Algebra is a mathematician's tool for solving problems. Algebra has 
applications to every human endeavor. From art to medicine to zoology, algebra can be a 
tool. People who say that they will never use algebra are people who do not know about 
algebra. Learning algebra is a bit like learning to read and write. If you truly leam 
algebra, you will use it. Knowledge of algebra can give you more power to solve 
problems and accomplish what you want in life. Algebra is a mathematicians’ shorthand! 

Algebraic Expressions 

An expression is a collection of numbers, variables, and +ve sign or -ve sign, of 
operations that must make mathematical and logical behaviour. 

For example 8x 2 + 9x -1 is an algebraic expression. 

What is Linear Algebra? 

One of the most important problems in mathematics is that of solving systems of linear 
equations. It turns out that such problems arise frequently in applications of mathematics 
in the physical sciences, social sciences, and engineering. Stated in its simplest terms, the 
world is not linear, but the only problems that we know how to solve are the linear ones. 
What this often means is that only recasting them as linear systems can solve non-linear 
problems. A comprehensive study of linear systems leads to a rich, formal structure to 
analytic geometry and solutions to 2x2 and 3x3 systems of linear equations learned in 
previous classes. 

It is exactly what the name suggests. Simply put, it is the algebra of systems of linear 
equations. While you could solve a system of, say, five linear equations involving five 
unknowns, it might not take a finite amount of time. With linear algebra we develop 
techniques to solve m linear equations and n unknowns, or show when no solution exists. 
We can even describe situations where an infinite number of solutions exist, and describe 
them geometrically. 

Linear algebra is the study of linear sets of equations and their transformation properties. 

Linear algebra, sometimes disguised as matrix theory, considers sets and functions, which 
preserve linear structure. In practice this includes a very wide portion of mathematics! 
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Thus linear algebra includes axiomatic treatments, computational matters, algebraic 
structures, and even parts of geometry; moreover, it provides tools used for analyzing 
differential equations, statistical processes, and even physical phenomena. 

Linear Algebra consists of studying matrix calculus. It fonnalizes and gives geometrical 
interpretation of the resolution of equation systems. It creates a formal link between 
matrix calculus and the use of linear and quadratic transformations. It develops the idea 
of trying to solve and analyze systems of linear equations. 

Applications of Linear algebra 

Linear algebra makes it possible to work with large arrays of data. It has many 
applications in many diverse fields, such as 

• Computer Graphics, 

• Electronics, 

• Chemistry, 

• Biology, 

• Differential Equations, 

• Economics, 

• Business, 

• Psychology, 

• Engineering, 

• Analytic Geometry, 

• Chaos Theory, 

• Cryptography, 

• Fractal Geometry, 

• Game Theory, 

• Graph Theory, 

• Linear Programming, 

• Operations Research 

It is very important that the theory of linear algebra is first understood, the concepts are 
cleared and then computation work is started. Some of you might want to just use the 
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computer, and skip the theory and proofs, but if you don’t understand the theory, then it 
can be very hard to appreciate and interpret computer results. 

Why using Linear Algebra? 

Linear Algebra allows for fonnalizing and solving many typical problems in different 
engineering topics. It is generally the case that (input or output) data from an experiment 
is given in a discrete form (discrete measurements). Linear Algebra is then useful for 
solving problems in such applications in topics such as Physics, Fluid Dynamics, Signal 
Processing and, more generally Numerical Analysis. 

Linear algebra is not like algebra. It is mathematics of linear spaces and linear functions. 
So we have to know the term "linear" a lot. Since the concept of linearity is fundamental 
to any type of mathematical analysis, this subject lays the foundation for many branches 
of mathematics. 

Objects of study in linear algebra 

Linear algebra merits study at least because of its ubiquity in mathematics and its 
applications. The broadest range of applications is through the concept of vector spaces 
and their transformations. These are the central objects of study in linear algebra 

1. The solutions of homogeneous systems of linear equations form paradigm 
examples of vector spaces. Of course they do not provide the only examples. 

2. The vectors of physics, such as force, as the language suggests, also provide 
paradigmatic examples. 

3. Binary code is another example of a vector space, a point of view that finds 
application in computer sciences. 

4. Solutions to specific systems of differential equations also fonn vector spaces. 

5. Statistics makes extensive use of linear algebra. 

6. Signal processing makes use of linear algebra. 

7. Vector spaces also appear in number theory in several places, including the 
study of field extensions. 

8. Linear algebra is part of and motivates much abstract algebra. Vector spaces 
form the basis from which the important algebraic notion of module has been 
abstracted. 
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9. Vector spaces appear in the study of differential geometry through the tangent 
bundle of a manifold. 

10. Many mathematical models, especially discrete ones, use matrices to represent 
critical relationships and processes. This is especially true in engineering as 
well as in economics and other social sciences. 

There are two principal aspects of linear algebra: theoretical and computational. A major 
part of mastering the subject consists in learning how these two aspects are related and 
how to move from one to the other. 

Many computations are similar to each other and therefore can be confusing without 
reasonable level of grasp of their theoretical context and significance. It will be very 
tempting to draw false conclusions. 

On the other hand, while many statements are easier to express elegantly and to 
understand from a purely theoretical point of view, to apply them to concrete problems 
you will need to “get your hands dirty”. Once you have understood the theory sufficiently 
and appreciate the methods of computation, you will be well placed to use software 
effectively, where possible, to handle large or complex calculations. 
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Course Segments 

The course is covered in 45 Lectures spanning over six major segments, which are given 
below; 

1. Linear Equations 

2. Matrix Algebra 

3. Determinants 

4. Vector spaces 

5. Eigen values and Eigenvectors, and 

6. Orthogonal sets 

Course Objectives 

The main purpose of the course is to introduce the concept of linear algebra, to explain 
the underline theory, the computational techniques and then try to apply them on real life 
problems. Mayor course objectives are as under; 

• To master techniques for solving systems of linear equations 

• To introduce matrix algebra as a generalization of the single-variable algebra of 
high school. 

• To build on the background in Euclidean space and formalize it with vector space 
theory. 

• To develop an appreciation for how linear methods are used in a variety of 
applications. 

• To relate linear methods to other areas of mathematics such as calculus and, 
differential equations. 
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Recommended Books and Supported Material 

I am indebted to several authors whose books I have freely used to prepare the lectures 
that follow. The lectures are based on the material taken from the books mentioned 
below. 

1. Linear Algebra and its Applications (3 rd Edition) by David C. Lay. 

2. Contemporary Linear Algebra by Howard Anton and Robert C. Busby. 

3. Introductory Linear Algebra (8 th Edition) by Howard Anton and Chris Rorres. 

4. Introduction to Linear Algebra (3 rd Edition) by L. W. Johnson, R.D. Riess and 
J.T. Arnold. 

5. Linear Algebra (3 rd Edition) by S. H. Friedberg, A.J. Insel and L.E. Spence. 

6. Introductory Linear Algebra with Applications (6 th Edition) by B. Kolrnan. 

I have taken the structure of the course as proposed in the book of David C. Lay . I would 
be following this book. I suggest that the students should purchase this book, which is 
easily available in the market and also does not cost much. For further study and 
supplement, students can consult any of the above mentioned books. 

I strongly suggest that the students should also browse on the Internet; there is plenty of 
supporting material available. In particular, I would suggest the website of David C. Lay; 
www. lavlina/gebra.com, where the entire material, study guide, transparencies are readily 
available. Another very useful website is www. wiIev.com/college/anton , which contains a 
variety of useful material including the data sets. A number of other books are also 
available in the market and on the internet with free access. 

I will try to keep the treatment simple and straight. The lectures will be presented in 
simple Urdu and easy English. These lectures are supported by the handouts in the form 
of lecture notes. The theory will be explained with the help of examples. There will be 
enough exercises to practice with. Students are advised to go through the course on daily 
basis and do the exercises regularly. 
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Schedule and Assessment 


The course will be spread over 45 lectures. Lectures one and two will be introductory and 
the Lecture 45 will be the summary. The first two lectures will lay the foundations and 
would provide the overview of the course. These are important from the conceptual point 
of view. I suggest that these two lectures should be viewed again and again. 

The course will be interesting and enjoyable, if the student will follow it regularly and 
completes the exercises as they come along. To follow the tradition of a semester system 
or of a term system, there will be a series of assignments (Max eight assignments) and a 
mid term exam. Finally there will be terminal examination. 

The assignments have weights and therefore they have to be taken seriously. 
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Lecture 2 
Background 

Introduction to Matrices 


Matrix A matrix is a collection of numbers or functions arranged into rows and columns. 

Matrices are denoted by capital letters A, B,..., Y, Z . The numbers or functions are called 
elements of the matrix. The elements of a matrix are denoted by small letters a,b,...,y,z. 

Rows and Columns The horizontal and vertical lines in a matrix are, respectively, called the 
rows and columns of the matrix. 

Order of a Matrix The size (or dimension) of matrix is called as order of matrix. Order of 
matrix is based on the number of rows and number of columns. It can be written as r x c ; r 
means no. of row and c means no. of columns. 


If a matrix has m rows and n columns then we say that the size or order of the matrix 
is m x « .If A is a matrix having m rows and n columns then the matrix can be written as 


A 


a n 
a 2 1 


V a m\ 


a \2 

a 22 

fi 

fi 


a 


m2 


fi. 

fi. 


fi. 


a ln 

a 2n 

fi. 

fi. 

a 


mn J 


The element, or entry, in the ith row and jth column of a m x n matrix A is written as a l 


For example: The matrix A = 
will be 2 x 3 


2 -1 3 
0 4 6 


has two rows and three columns. So order of A 


Square Matrix A matrix with equal number of rows and columns is called square matrix. 

U 7 -8^ 


For Example The matrix A = 


9 3 5 
1 -1 2 


has three rows and three columns. So it is a 


square matrix of order 3. 


Equality of matrices 

The two matrices will be equal if they must have 


©Virtual University Of Pakistan 


10 




2-Introduction to Matrices 


VU 


a) The same dimensions (i.e. same number of rows and columns) 

b) Corresponding elements must be equal. 



(4 7 

-8> 


OO 

1 

r- 


Example The matrices A = 

9 3 

[• -■ 

5 

2; 

and B = 

9 3 5 

h -i 2 j 

equal matrices 


(i.e A = B) because they both have same orders and same corresponding elements. 

Column Matrix A column matrix X is any matrix having n rows and only one column. 
Thus the column matrix X can be written as 

b 2 \ 

X — I b 2 \ —\bi\\ nx \ 


\pn\ J 

A column matrix is also called a column vector or simply a vector. 

Multiple of matrix A multiple of a matrix A by a nonzero constant k is defined to be 


kA = 


kan 

kci\ 2 

kai n 

ka 2 i 

ka 2 2 

ka 2n 

ka m\ 

ktini2 

kn 

u mn 


= [ty ] m x n 


Notice that the product kA is same as the product Ak . Therefore, we can write kA = Ak. 

It implies that if we multiply a matrix by a constant k, then each element of the matrix is to 
be multiplied by k. 

Example 1 


(a) 


5- 


" 2 

-3l 


"10 

-15' 

4 

-1 

= 

20 

-5 

1/5 

6 


1 

30 
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(b) 



' 1 ' 


' e' ' 


-2 

= 

-2e l 


4 


4e { 


Since we know that kA = Ak. Therefore, we can write 


1 

<N 

l_ 


1 

ro 

1 

<N 

l_ 


1 

(N 

i_ 

i 

fTl 

_1 


I 

1 

OJ 

1_ 


i 

L/\ 

1_ 


Addition of Matrices Only matrices of the same order may be added by adding 
corresponding elements. 

If A = \ajj ] and B = [by ] are two m x n matrices then A + B = [cty + by ] 

Obviously order of the matrix A + B is m x n 

Example 2 Consider the following two matrices of order 3x3 



f 2 

-1 

3 'l 


( 4 

7 

-8" 

A = 

0 

4 

6 

, B = 

9 

3 

5 


1-6 

10 

-5; 


l 1 

-1 

2 , 


Since the given matrices have same orders, therefore, these matrices can be added and their 
sum is given by 



( 2 + 4 

-1 + 7 

3 + (-8)^1 


r 6 

6 

-5" 

A + B = 

0 + 9 

4 + 3 

6 + 5 

= 

9 

7 

11 


6+i 

10 + (-1) 

-5 + 2 ) 


1-5 

9 

-3, 


Example 3 Write the following single column matrix as the sum of three column vectors 

(3t 2 -2e‘' 
t 2 + 7t 

\ 5t ; 

Solution 
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^ it 2 -2e^ 

t 2 +lt 
51 


f it 2 ^ 

t 2 

0 

V J 


+ 


'O' 

It 

\ 5t J 


+ 


A -2e^ 

0 
0 


V 


y 


'3' 

1 

v°y 


t 2 + 


'0' 

7 

v5y 


t + 


'-2' 

0 

vOy 


v y 

Difference of Matrices The difference of two matrices A and B of same order m x n is 
defined to be the matrix A - B = A + ( -B ) 

The matrix - B is obtained by multiplying the matrix B with -1. So that - B = ( -1 ) B 

Multiplication of Matrices We can multiply two matrices if and only if, the number of 
columns in the first matrix equals the number of rows in the second matrix. 

Otherwise, the product of two matrices is not possible. 

OR 

If the order of the matrix A is m x n then to make the product AB possible order of the 
matrix B must be «x p . Then the order of the product matrix AB is mx p . Thus 

A ■ R = C 

x n **nxp ^ m x p 


If the matrices A and B are given by 


Then 



«11 

a l2 ‘ 

a ln 


hi 

b \2 ‘ 

b \p 

A = 

a 2l 

a 22 ‘ 

a 2n 

, B = 

b 2\ 

b 22 ' 

b 2p 


_ a ml 

a m2 

a mn J 


b n\ 

b n2 

1 

Sf 


AB = 


«11 

a l2 ‘ 

a ln 

hi 

b l2 ‘ 

•• h P 

a 2l 

a 22 ‘ 

a 2n 

b 2l 

b 22 ‘ 

b 2p 


a ml a m2 ®mn J | b nl b n2 b np 


a n b lp +a l2 b 2p +&- + a hl b l 


np 


a \ A i +a l 2 b 2 \+--- + a Ul b n] fi 

U lA\ + a 22 b 2l + • • • + « 2 Al «2A p + a 22 b 2p + fi ' + a 2n h np 

\ fi fi 

a ,n A 1 + a m2 b 21 + ''' + At “rnAp + a m 2 b 2p + fi ' + “.nrPnp 
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X] a ik^kj 


U=1 


nx p 


Example 4 If possible, find the products AB and BA , when 
(a) A = 


r 4 7^ 


f9 -2" 


, B = 


v3 5; 


OC 

SO 


(b) 


A = 


35 83 

1 0 


v2 7y 


B = 


-4 -3 


v 2 0 y 


Solution (a) The matrices A and B are square matrices of order 2. Therefore, both of the 
products AB and BA are possible. 


AB = 


^4 7" 

"9 -23 


f4 • 9 + 7 • 6 4 • (-2) + 7-8") 


(78 48" 

v3 5 y 

v6 8 y 


^3 • 9 + 5 • 6 3 • (-2) + 5 • 8y 


l 57 34, 


Similarly 


BA = 


'9 -2" 

"4 73 


f9• 4 + (-2)• 3 9-7 + (-2)-53 


330 53"| 

v 6 8, 

v3 5; 


v 6 • 4 + 8 -3 6 • 7 + 8 • 5 J 


<N 

OO 

OO 


Note From above example it is clear that generally a matrix multiplication is not 
commutative i.e. AB ^ BA . 

(b) The product AB is possible as the number of columns in the matrix A and the number of 
rows in B is 2. However, the product BA is not possible because the number of column in the 
matrix B and the number of rows in A is not same. 


AB = 


5 8 
1 0 
2 7 


3-4 -3^ 

2 0 


r 5 • (-4) + 8-2 

5-(-3) + 8-03 


3-4 

-15" 

1 • (-4) + 0-2 

1 • (-3) + 0-0 

= 

-4 

-3 

v 2 • (-4) + 7-2 

2 • (-3) + 7 • Oy 


l 6 

“by 



OO 

00 

r- 

\ 1 

330 53 " 

AB = 


S 

11 



^57 34 ) 

1 1 

<N 

OO 

OO 

'xl- 


Clearly AB BA. 
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AB = 


r -4 -15^ 

-4 -3 


v 


-6 


However, the products is not possible. 

Example 5 



(2 -1 3" 

(— 3^ 


r 2 • (-3) + (-1) • 6 + 3 • 4^1 


r o" 

(a) 

0 4 5 

6 

= 

0 • (-3) + 4 • 6 + 5 • 6 

= 

44 


l 1 9 y 

V 4 2 


[l' (-3) + (-7) • 6 + 9 • 4J 


1-9, 


r -A 2^ 

fx) 


'-4x + 2 / 

v3 8, 



v 3x + 8y , 


(b) 


Multiplicative Identity For a given any integer n , the n x n matrix 

r \ 0 0 ••• 0^ 

0 1 0 ••• 0 
7 = 1 0 0 1 ••• 0 


v 0 0 0 ••• 1 J 


is called the multiplicative identity matrix. If ^4 is a matrix of order n x n , then it can be 
verified that / • A - A-1 = A 

d o 


Example / = 


f \ Cf 

v 0 1, 


, / = 


0 1 0 
0 0 1 


are identity matrices of orders 2x2 and 3x3 


respectively and If B = 


9 -2 
6 8 


then we can easily prove that BI = IB = B 


©Virtual University Of Pakistan 


15 




2-Introduction to Matrices 


VU 


Zero Matrix or Null matrix A matrix whose all entries are zero is called zero matrix or null 
matrix and it is denoted by O. 

f 0 0 ) 






r 0 0" 

For example 

0 = 


; o = 






v o o y 


and so on. If A and O are the matrices of same orders, then A + O - O + A - A 

Associative Law The matrix multiplication is associative. This means that if A, B and 
C ar em x p , pxr and rxn matrices, then A(BC) = ( AB)C 

The result is a m x n matrix. This result can be verified by taking any three matrices which 
are confirmable for multiplication. 


Distributive Law If B and C are matrices of order r x n and A is a matrix of order m x r , 
then the distributive law states that 

A(B + C) = AB + AC 

Furthermore, if the product (A + B)C is defined, then 

(.A + B)C = AC + BC 


Remarks 

It is important to note that some rules arithmetic for real numbers M do not carry over the 
matrix arithmetic. 

For example, \/a,b,c and d e M. 

i) if ab = cd and a ± 0, then b = c (Law of Cancellation) 

ii) if ab = 0 , then least one of the factors a or b (or both) are zero. 

However the following examples shows that the corresponding results are not true in case of 
matrices. 

Example 



'0 ll 


“l ll 


“2 

5“ 


“l 

7“ 

Let A = 

0 2 

, B = 

3 4 

, c = 

_3 

4 

and D = 

0 

0 


then one can easily check that 


AB = AC = 


3 4 
6 8 


. But B i^C. 


Similarly neither A nor B are zero matrices but AD = 


0 

0 


0 

0 


But if D is diagonal say D = 


1 0 
0 7 


then AD ^ DA . 


Determinant of a Matrix Associated with every square matrix A of constants, there is a 
number called the detenninant of the matrix, which is denoted by det(A) or 4 . There is a 
special way to find the determinant of a given matrix. 
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Example 6 Find the detenninant of the following matrix A = 


Solution The detenninant of the matrix A is given by 

3 6 2| 


r l 6 2^ 
2 5 1 


v-1 2 4 y 


det(A) = 


2 5 1 


|-1 2 41 

We expand the dct(/l) by first row, we obtain 

3 6 2| 


det(A) = 


=3 


5 1 
2 4 


-6 


2 1 
-1 4 


+2 


or 


2 5 1 
|-1 2 4 

det(A) = 3(20 - 2) - 6(8 +1) + 2(4 + 5) = 18 


2 5 

-1 2 


• • tv* • 

Transpose of a Matrix The transpose of m x n matrix A is denoted by A and it is 
obtained by interchanging rows of A into its columns. In other words, rows of A become the 

columns of A n . Clearly A" is n x m matrix. 


If A = 

a n a \2 

a 2\ a 22 

■■ a in ' 

•• a 2n 

A tr 

. then 

a \ 1 a 2\ 

a \2 a 22 

•• a ,n\ ' 

•• a m2 


\ a m\ a m2 

•• a t 

mn J 


\ a \n a 2 n 

•• a 

mn J 


• • • • tv* * 

Since order of the matrix A is mxn , the order of the transpose matrix A is n x m . 

Properties of the Transpose 

The following properties are valid for the transpose; 

• The transpose of the transpose of a matrix is the matrix itself:' 

• The transpose of a matrix times a scalar ( k ) is equal to the constant times the 
transpose of the matrix: ( ABC) 1 = C T B r A T ( kA) T = kA T 

• The transpose of the sum of two matrices is equivalent to the sum of their 
transposes: (A + B) T =A T +B T 

• The transpose of the product of two matrices is equivalent to the product of their 
transposes in reversed order: ( AB) r = B T A r 

• The same is true for the product of multiple matrices: ( ABC) T = C T B r A T 
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Example 7 (a) The transpose of matrix A = 


' 3 

6 

2^ 


r 3 

2 

-f| 

2 

5 

1 

is A T = 

6 

5 

2 

v-1 

2 

4, 


^2 

1 

4 j 


(b) If X = 




v3y 


, then X T = [5 0 3] 


Multiplicative Inverse Suppose that A is a square matrix of order n x n . If there exists an 
n x n matrix B such that AB = BA = I , then B is said to be the multiplicative inverse of the 

matrix A and is denoted by B = A~ l . 

(1 4 ^ 

For example: If A = 


v 2 10 , 


then the matrix B = 


(5 

V 


-2 


-1 1/2 


is multiplicative inverse of A 


"1 4 ^ 

r 5 -2 ^ 


r \ 0^ 

v 2 10, 

V-l 1/2; 


V 0 1, 


because AB 
Similarly we can check that BA = I 


Singular and Non-Singular Matrices A square matrix A is said to be a non-singular 
matrix ifdet(y4)^0, otherwise the square matrix A is said to be singular. Thus for a 
singular matrix A we must have det(^) = 0 


Example: A = 


2 

1 

2 


3 

1 

-3 


-1 

0 

5 


\A\ = 2(5 — 0) -3(5-0) —1(—3 —2) 
= 10 -15 +5 =0 


which means that A is singular. 

Minor of an element of a matrix 

Let A be a square matrix of order n x n. Then minor AT of the element a tj e/I is the 
determinant of (n -1) x (n - 1) matrix obtained by deleting the ith row and jth column 
from A. 
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Example If A = 


M n and is defined to be M u 


2 3-1 

1 1 0 

2-3 5 

1 0 
2 5 

Cofactor of an element of a matrix 


is a square matrix. The Minor of 3 e A is denoted by 


5-0 = 5 


Let A be a non singular matrix of order n x n and let C .. denote the cofactor (signed minor) 
of the corresponding entry ay e A , then it is defined to be Cy = (-\) ,+J My 
"2 3 -1 

is a square matrix. The cofactor of 3 e A is denoted by 


Example If A = 


1 1 

2 -3 


C 12 and is defined to be C 12 = = (-1) 


1+2 


1 0 
2 5 


(5 - 0) = -5 


Theorem If A is a square matrix of order n x n then the matrix has a multiplicative inverse 


A 1 if and only if the matrix A is non-singular. 


Theorem Then inverse of the matrix A is given by A 1 = \ (C ; y )^ 


dcl(A) 


1. For further reference we take n — 2 so that A is a 2x2 non-singular matrix given by 


A = 


Si «12 A 


V°21 a 22 j 

Therefore Cjj =«22’ Q 2 = _<7 2L C 21 =_a 12 andC 22 =a\\- So that 


+-■= 1 


det(2l) 


2. For a 3 x 3 non-singular matrix A= 


^ a 22 

_<3 21^ 

tr 

1 

( a 22 

~ a \2^ 

a \2 

a n ) 


det(.d) 

V~ a 2\ 

a \\ J 

Si 

a \2 

a n ^ 




a 2\ 

a 22 

a 23 




V a 31 

a 32 

a 33 J 





©Virtual University Of Pakistan 


19 




2-Introduction to Matrices 


Qi = 


a 22 a 23 
a 32 a 33 


C 12 = ~ 


a 21 a 23 
a 3\ a 33 


r = 

9 ^ 13 


Therefore, inverse of the matrix A is given by A 1 = 


det^4 


21 a 22 

SI a 7>2 

and so on. 

C n 

c 

'-'21 

c ^ 

'-"31 

C u 

c 

^22 

c 

.32 


r 

'-'23 

c 

'-'33 J 


f 1 4 ^ 


Example 8 Find, if possible, the multiplicative inverse for the matrix A = 
Solution The matrix A is non-singular because dci(A) = 


v 2 10 , 


1 4 

2 10 


10-8 = 2 


4 - 1 . 


1 

1 

o 


( 5 -2" 

2 

1-2 1 ) 


l-l 1/2, 


Check 


AA~ l = 


AA~ l = 


"1 4 N 

" 5 -2" 


"5-4 -2 + 2^ 


fl 0" 

v 2 10, 

v-1 1/2, 


(,10-10 -4 + 5; 


10 1, 


" 5 -2^ 

"1 4 f 


© 

<N 

1 

O 

<N 

1 

SO 


fl 0" 

v-1 1/2, 

v2 10; 


,-1 + 1 -4 + 5; 


v0 1, 


= / 


= / 


Example 9 Find, if possible, the multiplicative inverse of the following matrix 

f 2 2^ 


A = 


V3 3 y 


Solution The matrix is singular because 


det(yf) = 


2 2 


3 3 

- 1 . 


= 2- 3- 2- 3 = 0 


Therefore, the multiplicative inverse A of the matrix does not exist. 

Example 10 Find the multiplicative inverse for the following matrix 

( 2 2 0 ^ 


A= 


-2 1 1 
3 0 1 


vu 
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Solution Since det(yl) = 


2 2 0 
-2 1 1 
3 0 1 


= 2(l-0)-2(-2-3) + 0(0-3) = 12*0 


Therefore, the given matrix is non singular. So, the multiplicative inverse A 1 of the matrix 
A exists. The cofactors corresponding to the entries in each row are 


Qt = 


1 1 



-2 1 



=1, 

C 12 =- 


-5, C 13 - 

0 1 



3 1 




2 0 


2 0 


C 2\ =~ 


= -2, C 22 = 


= 2, C 23 =- 


0 1 


3 1 



-2 1 

3 0 

2 2 

3 0 


= -3 


= 6 



2 0 



2 0 


2 2 

C 3 l = 

1 1 

= 2, 

<^32 

-2 1 

= -2, C 33 = 

-2 1 


= 6 


Hence 


A-=± 

12 


' 1 

-2 

2 'l 


r 1/12 

-1/6 

1/6 ' 

5 

2 

-2 

= 

5/12 

1/6 

-1/6 

v-3 

6 

6 2 


[-1/4 

1/2 

1/2 y 


We can also verify that A- A 1 = A 1 • A = I 


Derivative of a Matrix of functions 


Suppose that 


A(t)= ciy (t) 


JfflX« 

is a matrix whose entries are functions those are differentiable in a common interval, then 
derivative of the matrix A(t) is a matrix whose entries are derivatives of the corresponding 
entries of the matrix A(t) . Thus 

dA 


dt 


day 

dt 


m x n 


The derivative of a matrix is also denoted by A'(t). 

Integral of a Matrix of Functions 

Suppose that A(t) = \a i; (t )) is a matrix whose entries are functions those are continuous 

v ‘J 'mxn 

on a common interval containing!, then integral of the matrix A(t) is a matrix whose entries 
are integrals of the corresponding entries of the matrix A(t ). Thus 

i x . x 


| A(s)ds = ajj(s)ds 


' mxn 
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sin2? 


Example 11 Find the derivative and the integral of the following matrix X(t)= e~ 


Solution The derivative and integral of the given matrix are, respectively, given by 


— (sin 2 ?) / 2cos2f' 

X'(t)= — (e 3t ) = 3e 3t 

dt 

— (8f — 1) V 8 , 

dt ) 


i 

| sin 2 sds 


-l/2cos2f + l/2 


and Jx(s)r/s = J e" ds = 1 / 3e 3/ —1/3 


Exercise 

Write the given sum as a single column matrix 

f _1 l ( 3t ] 

L 3 1 t +(/-!) -t -2 4 


-n 2 


1-3 4 V t 


2. 2 5 -1 2t-\ +1-8 


0 -4 -2 -t 


4 -6 


4 t -t 


i 

J 8s - Ids 


Determine whether the given matrix is singular or non-singular. If singular, find A 
r 3 2 P 

3. A= 4 1 0 

v-2 5 -1, 

'4 1 -r 

4. A= 6 2-3 


-2 -1 2 


Find 

dt 


r (— sin2t - 4cos2i 
5 - X= 2 

- 3sin2t + 5cos2i 


6. If A(t) = 


4 1 

e cos nt 


It 3t -1 


7. Find the integralJ B(t)dt if B(t) = 

1 . 1 


z 1 

then find (a) J A(t)dt , (b) J A(s)ds. 


6 1 2 
Ut 41 


©Virtual University Of Pakistan 


22 



3-System of Linear Equations 


VU 


Lecture 3 

Systems of Linear Equations 

In this lecture we will discuss some ways in which systems of linear equations arise, how 
to solve them, and how their solutions can be interpreted geometrically. 

Linear Equations 

We know that the equation of a straight line is written as y = mx + c , where m is the 
slope of line(Tan of the angle of line with x-axis) and c is the y-intercept(the distance at 
which the straight line meets y-axis from origin). 

Thus a line in R 2 (2-dimensions) can be represented by an equation of the form 
a x x + a 2 y =b (where a\, a 2 not both zero). Similarly a plane in R 3 (3-dimensional space) 

can be represented by an equation of the fonn a 1 x + a 2 y + a 3 z = b (where a\, ai, a 3 not 
all zero). 

A linear equation in n variables X l ,X 2 ,---,X n can be expressed in the form 
ctjjCj + a 2 x 2 H-h a n x n = b (hyper plane in M” )-(1) 


where a l ,a 2 ,-",a n and b are constants and the “a’s” are not all zero. 

Homogeneous Linear equation 

In the special case if b = 0, Equation (1) has the fonn ct\X x + a 2 x 2 -\— + a n x n = 0 (2) 

This equation is called homogeneous linear equation. 

Note A linear equation does not involve any products or square roots of variables. All 
variables occur only to the first power and do not appear, as arguments of trigonometric, 
logarithmic, or exponential functions. 

Examples of Linear Equations 

(1) The equations 

2x x + 3x 2 + 2 = x 3 and x 2 = 2 (V5 + Xj) + 2x 3 are both linear 

(2) The following equations are also linear 

x + 3y = 7 Xj - 2x 2 - 3x 3 + x 4 = 0 

jx-y + 3z = -l x, + x 2 4- hx n =1 

(3) The equations 3x, - 2x 2 = x t x 2 and x 2 = 4 «Jx x - 6 

are not linear because of the presence ofXjX, in the first equation and Jy in the second. 
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System of Linear Equations 

A finite set of linear equations is called a system of linear equations or linear system. The 
variables in a linear system are called the unknowns. 

For example, 

4x l -x 2 + 3x 3 = -1 
3xj +x 2 + 9x 3 = -4 

is a linear system of two equations in three unknowns xi, xo, and X 3 . 


General System of Linear Equations 

A general linear system of m equations in n-unknowns x l , X 2 , • • •, X n can be written as 
a n x j +a 12 x 2 -1 -t-a ln x„ =b x 


a 21 Xj + a 22 x 2 -\ - 1 - a 2n x n = b 2 


(3) 


a ,x, + a,„,x, -I-f- a x =b 

mi 1 ml 1 mn n m 


Solution of a System of Linear Equations 

A solution of a linear system in the unknowns Xj, X 2 , • • • , X n is a sequence of n numbers 
s t ,s 2 ,---,s n such that when substituted for X, , X 2 , • • • , X„ respectively, makes every 
equation in the system a true statement. The set of all such solutions { 1 s , 1 ,s 2 ,---,,s B }of a 
linear system is called its solution set. 

Linear System with Two Unknowns 

When two lines intersect in R 2 , we get system of linear equations with two unknowns 

a,x + b x v = q 

For example, consider the linear system 

a 2 x + b 2 y = c 2 


The graphs of these equations are straight lines in the xy-plane, so a solution (x, v) of this 

system is infact a point of intersection of these lines. 

Note that there are three possibilities for a pair of straight lines in xy-plane: 

1. The lines may be parallel and distinct, in which case there is no intersection and 
consequently no solution. 

2. The lines may intersect at only one point, in which case the system has exactly 
one solution. 

3. The lines may coincide, in which case there are infinitely many points of 
intersection (the points on the common line) and consequently infinitely many 
solutions. 
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Consistent and inconsistent system 

A linear system is said to be consistent if it has at least one solution and it is called 
inconsistent if it has no solutions. 

Thus, a consistent linear system of two equations in two unknowns has either one 
solution or infinitely many solutions - there is no other possibility. 

Example consider the system of linear equations in two variables 
x l -2x 2 =-l, -x 1 +3x 2 =3 
Solve the equation simultaneously: 

Adding both equations we get x 2 = 2, Put x 2 = 2 in any one of the above equation we 
getXj = 3. So the solution is the single point (3, 2). See the graph of this linear system 


*2 



See the graphs to the following linear systems: 


(a) x 1 -2x 2 =-l 

- Xj + 2x 2 = 3 


( b ) Xj-2x,=-l 

- Xj + 2 x 2 = 1 


*2 



(a) No solution. 



©Virtual University Of Pakistan 


25 




3-System of Linear Equations 


VU 


Linear System with Three Unknowns 

Consider r a linear system of three equations in three unknowns: 
a x x + b x y + qz = d x 

a 2 x + b 2 y + c x z = d 2 
a 2 x + b 2 y + c 2 z = d 2 

In this case, the graph of each equation is a plane, so the solutions of the system, If any 
correspond to points where all three planes intersect; and again we see that there are only 
three possibilities - no solutions, one solution, or infinitely many solutions as shown in 
figure. 



Theorem 1 Every system of linear equations has zero, one or infinitely many solutions; 
there are no other possibilities. 

x- v = 1 

Example 1 Solve the linear system 

2x + y = 6 

Solution 

7 

Adding both equations, we getx = —. Putting this value of x in 1st equation, we 

4 7 4 

get y = — . Thus, the system has the unique solution x = —,y = —. 


Geometrically, this means that the lines represented by the equations in the system 

f 7 4^1 

and thus has a unique solution. 


intersect at a single point 


3 3 


Example 2 Solve the linear system 

3x + 3y = 6 

Solution 

Multiply first equation by 3 and then subtract the second equation from this. We obtain 
0 = 6 

This equation is contradictory. 
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Geometrically, this means that the lines corresponding to the equations in the original 
system are parallel and distinct. So the given system has no solution. 

4x - 2 v = 1 

Example 3 Solve the linear svstein 
-- 16x-8y = 4 


Solution 


Multiply the first equation by -4 and then add in second equation. 

-16x + 8v = -4 
16x-8v = 4 
0 ^0 

Thus, the solutions of the system are those values of x and y that satisfy the single 
equation 4x - 2y = 1 

Geometrically, this means the lines corresponding to the two equations in the original 
system coincide and thus the system has infinitely many solutions. 

Parametric Representation 

It is very convenient to describe the solution set in this case is to express it 
parametrically. We can do this by letting y = t and solving for jc in terms of t, or by 
letting x = t and solving for y in terms of t. 

The first approach yields the following parametric equations (by taking y=t in the 
equation 4x - 2y = 1) 


4x - 2t = 1, y = t 


x = — +—t, y = t 
4 2 

We can now obtain some solutions of the above system by substituting some numerical 
values for the parameter. 

1 3 

Example For t = 0 the solution is (—,0). For t = 1, the solution is (—,1) and for t = - 1 


the solution is (—, -1) etc. 

4 


x- y + 2z = 5 

Example 4 Solve the linear system 2x - 2y + 4z = 10 

3x-3y + 6z = 15 
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Solution 

Since the second and third equations are multiples of the first. 

Geometrically, this means that the three planes coincide and those values of x, y and z 
that satisfy the equation x - y + 2z = 5 automatically satisfy all three equations. 

We can express the solution set parametrically as 

x = 5 + t l -2t 2 ,y = t l , z — t 2 

Some solutions can be obtained by choosing some numerical values for the parameters. 

For example if we take y = t x = 2 and z = t 2 = 3 then 
x = 5 + t l -2t 2 
= 5+ 2-2(3) 

= 1 

Put these values of x, y, and z in any equation of linear system to verify 

x- y + 2z = 5 
1-2 +2(3) = 5 
1-2 + 6 = 5 
5 = 5 

Hence x = 1, y = 2, z = 3 is the solution of the system. Verified. 

Matrix Notation 

The essential infonnation of a linear system can be recorded compactly in a rectangular 
array called a matrix. 


Xj - 2 x 2 + x 3 = 0 

Given the system 2.x, - 8x 3 = 8 

-4xj + 5x, + 9 x 3 = -9 

With the coefficients of each variable aligned in columns, the matrix 


1 

0 

-4 


is called the coefficient matrix (or matrix of coefficients) of the system. 


-2 1 
2 -8 
5 9 


An augmented matrix of a system consists of the coefficient matrix with an added column 
containing the constants from the right sides of the equations. It is always denoted by Ab 
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Ab - 


1 

0 

-4 


-2 1 0 
2-8 8 
5 9-9 


Solving a Linear System 

In order to solve a linear system, we use a number of methods. 1st of them is given 
below. 

Successive elimination method In this method the term in the first equation of a 
system is used to eliminate the Xj terms in the other equations. Then we use thex, term 
in the second equation to eliminate the x 7 terms in the other equations, and so on, until 
we finally obtain a very simple equivalent system of equations. 


Xj - 2x 2 + x 3 = 0 

Example 5 Solve 2x 2 - 8x 3 = 8 

-4xj + 5 x 2 + 9x 3 = -9 


Solution We perform the elimination procedure with and without matrix notation, 
and place the results side by side for comparison: 


Xj - 2 x 2 + x 3 = 0 
2x 2 - 8x 3 = 8 
-4xj + 5 x 2 + 9x 3 = -9 


1-210 
0 2-88 
-4 5 9 -9 


To eliminate the x, term from third equation add 4 times equation 1 to equation 3, 

4xj - 8x 2 + 4x 3 = 0 
-4xj + 5 x 2 + 9x 3 = -9 
-3x 2 + 13x 3 = -9 


The result of the calculation is written in place of the original third equation: 


x, 


2x 2 + x 3 = 0 


2x, 


8x 3 = 8 


-3x 2 + 13x 3 = 


1-210 
0 2-88 
0 -3 13 -9 


Next, multiply equation 2 by V 2 in order to obtain 1 as the coefficient forx 7 
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x, — 2x 2 + x 3 = 0 
x 2 — 4 x 3 = 4 
— 3 x 2 +1 3x 3 = —9 


1-210 
0 1-44 

0 -3 13 -9 


To eliminate the x 2 term from third equation add 3 times equation 2 to equation 3, 


The new system has a triangular form 
Xj — 2 x 2 + x 3 = 0 
x 2 — 4x 3 = 4 
x 3 = 3 


1-210 
0 1-44 

0 0 13 


Now using 3 rd equation eliminate theXj tenn from first and second equation i.e. multiply 
3 ld equation with 4 and add in second equation. Then subtract the third equation from first 
equation we get 


Xj — 2 x 2 = —3 
x 2 = 16 
x 3 = 3 


1-2 0-3 
0 1 0 16 
0 0 13 


Adding 2 times equation 2 to equation 1, we obtain the result 


x 1 = 29 

"1 

0 

0 

29 

so 

II 

(N 

X 

0 

1 

0 

16 

x 3 =3 

0 

0 

1 

3 


This completes the solution. 

Our work indicates that the only solution of the original system is (29, 16, 3). 

To verify that (29, 16, 3) is a solution, substitute these values into the left side of the 
original system for xi, X 2 and x 3 and after computing, we get 

(29)-2(16)+ (3) = 29 - 32 + 3 = 0 
2(16)-8(3) = 32-24 = 8 
-4(29)+ 5(16)+ 9(3) =-116 + 80 + 27 = —9 

The results agree with the right side of the original system, so (29, 16, 3) is a solution of 
the system. 
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This example illustrates how operations on equations in a linear system correspond to 
operations on the appropriate rows of the augmented matrix. The three basic operations 
listed earlier correspond to the following operations on the augmented matrix. 


Elementary Row Operations 

1. (Replacement) Replace one row by the sum of itself and a nonzero multiple of 
another row. 

2. (Interchange) Interchange two rows. 

3. (Scaling) Multiply all entries in a row by a nonzero constant. 

Row equivalent matrices 

A matrix B is said to be row equivalent to a matrix A of the same order if B can be 
obtained from A by perfonning a finite sequence of elementary row operations of A. 

If A and B are row equivalent matrices, then we write this expression mathematically as 
A ~B. 



“ 1 

-2 

1 

0 ' 


“l -2 

1 

0 ' 


For example 

0 

2 

-8 

8 

~ 

0 2 

-8 

8 

are row equivalent matrices 


-4 

5 

9 

-9 


0 -3 

13 

-9 



because we add 4 times of 1 st row in 3 ld row in 1 st matrix. 


Note If the augmented matrices of two linear systems are row equivalent, then the two 
systems have the same solution set. 

Row operations are extremely easy to perfonn, but they have to be learnt and practice. 

Two Fundamental Questions 

1. Is the system consistent; that is, does at least one solution exist? 

2. If a solution exists is it the only one; that is, is the solution unique? 

We try to answer these questions via row operations on the augmented matrix. 

Example 6 Determine if the following system of linear equations is consistent 

Xj - 2x 2 + x 3 = 0 

2x 2 - 8x 3 = 8 
-4xj + 5 x 2 + 9x 3 = -9 


Solution 


First obtain the triangular matrix by removing x 1 and X? term from third equation and 
removing X 2 from second equation. 


©Virtual University Of Pakistan 


31 



First divide the second equation by 2 we get 

Xj - 2x, + x 3 = 0 1-2 1 0 

x 2 - 4x 3 =4 0 1-44 

-4xj + 5 x 2 + 9x 3 = -9 -4 5 9 -9 

Now multiply equation 1 with 4 and add in equation 3 to eliminate x / 
from third equation. 

Xj - 2 x 2 + x 3 = 0 1-2 1 0 

x 2 - 4 x 3 =4 0 1-44 

- 3x, +1 3 x 3 = -9 0-313-9 

Now multiply equation 2 with 3 and add in equation 3 to eliminate X? 
from third equation. 

Xj — 2 x 2 + x 3 = 0 1-210 

x 2 — 4 x 3 =4 0 1-44 

x 3 — 3 [o 0 1 3 

Put value of X j in second equation we get 
x 2 - 4(3) = 4 
x 2 = 16 

Now put these values of x 2 and x 3 in first equation we get 
Xj -2(16)+ 3 = 0 
Xj = 29 

So a solution exists and the system is consistent and has a unique solution. 

Example 7 Solve if the following system of linear equations is consistent. 

x 2 - 4 x 3 = 8 
2xj - 3 x 2 + 2x 3 = 1 
5xj -8 x 2 +7x 3 = 1 
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Solution The augmented matrix is 

"0 1 -4 8“ 

2-321 

5-871 

To obtain Xj in the first equation, interchange rows 1 and 2: 

"2 -3 2 f 

0 1-48 

5-871 

To eliminate the 5x / term in the third equation, add -5/2 times row 1 to row 3: 

"2-3 2 1 1 

0 1-48 

0 -1/2 2 -3/2 

Next, use the x? term in the second equation to eliminate the -(1/2) x? term from the 

third equation. Add 'A times row 2 to row 3: 

[2-32 1 

0 1-48 

00 05/2 

The augmented matrix is in triangular form. 

To interpret it correctly, go back to equation notation: 

2xj - 3 x 2 + 2x 3 = 1 
x 2 - 4x 3 = 8 

0 = 2.5 

There are no values of X;, X 2 , X 3 that will satisfy because the equation 0 = 2.5 is never 

true. 

Hence original system is inconsistent (i.e., has no solution). 


©Virtual University Of Pakistan 


33 




3-System of Linear Equations 


VU 


Exercises 

1. State in words the next elementary “row” operation that should be performed on the 
system in order to solve it. (More than one answer is possible in (a).) 


a. x, + 4x 2 - 2x 3 + 8x 4 = 12 
x 2 - 7x 3 + 2x 4 = -4 
5x 3 - x 4 = 7 
x 3 + 3x 4 = -5 


b. Xj - 3x 2 + 5x 3 - 2x 4 = 0 

x 2 + 8x 3 = -4 

2x 3 = 7 

x 4 = 1 


2. The augmented matrix of a linear system has been transformed by row operations into 
the fonn below. Determine if the system is consistent. 

“15 2 - 6 “ 

0 4-72 
0 0 5 0 


3. Is (3, 4, -2) a solution of the following system? 

5xj - x 2 + 2x 3 = 7 
-2xj + 6x 2 + 9x 3 = 0 
-7xj + 5x 2 - 3x 3 = -7 

4. For what values of h and k is the following system consistent? 

2xj - x 2 = h 
-6xj + 3 x 2 = k 

Solve the systems in the exercises given below; 



x 2 + 5 x 3 = -4 


Xj - 5 x 2 + 4x 3 = -3 

5. 

Xj + 4 x 2 + 3x 3 = -2 

6. 

2xj -7 x 2 +3x 3 = -2 


2x 1 +7x 2 + x 3 = — 1 


2xj - x 2 -7x 3 = 1 


7. 


Xj + 2 x 2 = 4 

Xj-3x 2 -3x 3 =2 8. 

x 2 + x 3 = 0 


2xj -4 x 3 = -10 
x 2 + 3x 3 = 2 
3xj + 5 x 2 + 8x 3 = -6 
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Determine the value(s) of h such that the matrix is augmented matrix of a consistent 
linear system. 



n -3 h i 


" 1 h -2 

9. 

1 

SO 

<N 

1 

_1 

10. 

-4 2 10 


Find an equation involving g, h, and that makes the augmented matrix correspond to a 
consistent system. 



~ 1 

-4 

7 

g 


~ 2 

5 

-3 

g 

11. 

0 

3 

-5 

h 

12. 

4 

7 

-4 

h 


-2 

5 

-9 

k 


-6 

-3 

1 

k 


Find the elementary row operations that transfonn the first matrix into the second, and 
then find the reverse row operation that transfonns the second matrix into first. 



“l 

3 

-1] 


“l 

3 

-ll 


“0 

5 

-3] 


“ 1 

5 

-2' 

13. 

0 

2 

-4 


0 

1 

-2 

14. 

1 

5 

-2 

n 

0 

5 

-3 


0 

-3 

4 


0 

-3 

4 


2 

1 

8 


2 

1 

8 



L J L 

j 


"13-1 51 


"13-1 5" 

15. 

0 1-42 


0 1-42 


0 2-5 -1 


0 0 3 -5 
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Lecture 4 

Row Reduction and Echelon Forms 

To analyze system of linear equations, we shall discuss how to refine the row reduction 
algorithm. While applying the algorithm to any matrix, we begin by introducing a non 
zero row or column (i.e. contains at least one nonzero entry) in a matrix, 

Echelon form of a matrix 


A rectangular matrix is in echelon form (or row echelon form) if it has the following three 
properties: 

1. All nonzero rows are above any rows of all zeros 

2. Each leading entry of a row is in a column to the right of the leading entry of the 
row above it. 

3. All entries in a column below a leading entry are zero. 


Reduced Echelon Form of a matrix 


If a matrix in echelon form satisfies the following additional conditions, then it is in 
reduced echelon form (or reduced row echelon form): 

4. The leading entry in each nonzero row is 1. 

5. Each leading 1 is the only nonzero entry in its column. 

Examples of Echelon Matrix form 

The following matrices are in echelon form. The leading entries (°) may have any 
nonzero value; the started entries (*) may have any values (including zero). 


2-3 2 1 

1. 0 1 -4 8 

00 05/2 


©Virtual University Of Pakistan 


36 




4-Row Reduction and Echelon Forms 


VU 








"0 

o 

* 

* 

* 

* 

* 

* 

* 

* 


O 

* 

* 

* 


0 

0 

0 

o 

* 

* 

* 

* 

* 

* 


0 

o 

* 

* 












2. 

0 

0 

0 

0 


3. 

0 

0 

0 

0 

o 

* 

* 

* 

* 

* 







0 

0 

0 

0 

0 

o 

* 

* 

* 

* 


0 

0 

0 

0 


















0 

0 

0 

0 

0 

0 

0 

0 

o 

* 


"l 

4 

-3 

7" 





'l 

1 

o' 





4. 

0 

1 

6 

2 




5. 

0 

1 

0 






0 

0 

1 

5 





0 

0 

0 






"0 

1 

2 

6 

O ' 











6. 

0 

0 

1 

-1 

0 












0 

0 

0 

0 

1 












Examples of Reduced Echelon Form 

The following matrices are in reduced echelon form because the leading entries are l’s, 
and there are 0’s below and above each leading 1. 



"1 

0 

0 

29 














1 . 

0 

1 

0 

16 















0 

0 

1 

1 






















"0 

1 

* 

0 

0 

0 

* 

* 

0 

*“ 


'1 

0 

* 

*“ 




0 

0 

0 

1 

0 

0 

* 

* 

0 

* 


0 

1 

* 

* 














2. 

0 

0 

0 

0 



3. 

0 

0 

0 

0 

1 

0 

* 

* 

0 

* 









0 

0 

0 

0 

0 

1 

* 

* 

0 

* 


0 

0 

0 

0 






















0 

0 

0 

0 

0 

0 

0 

0 

1 

* 













"0 

1 

-2 

0 

1 " 



1 

0 

0 

4~ 


'1 

0 

o ' 




0 

0 

0 

1 

3 


4. 

0 

1 

0 

7 

5. 

0 

1 

0 



6 . 

0 

0 

0 

0 

0 



0 

0 

1 

-1 


0 

0 

1 










L 








- 1 




0 

0 

0 

0 

0 _ 



Note A matrix may be row reduced into more than one matrix in echelon form, using 
different sequences of row operations. However, the reduced echelon form obtained from 
a matrix, is unique. 

Theorem 1 (Uniqueness of the Reduced Echelon Form) Each matrix is row equivalent 
to one and only one reduced echelon matrix. 
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Pivot Positions 

A pivot position in a matrix A is a location in A that corresponds to a leading entry in an 
echelon form of A 

Note When row operations on a matrix produce an echelon form, further row operations 
to obtain the reduced echelon form do not change the positions of the leading entries. 

Pivot column 

A pivot column is a column of A that contains a pivot position. 

Example 2 Reduce the matrix A below to echelon form, and locate the pivot columns 
[ 0 -3 -6 4 9 1 

-1 -2-13 1 

A = 

-2 -3 0 3 -1 

1 4 5-9-7 

Solution Leading entry in first column of above matrix is zero which is the pivot 
position. A nonzero entry, or pivot, must be placed in this position. So interchange first 
and last row. 

"l J Pivot 4 5 -9 -7 

-1 -2-13 1 

-2 -3 0 3 -1 

0 -3-6 4 9 

^ Pivot Column 

Since all entries in a column below a leading entry should be zero. For this add row 1 in 
row 2, and multiply row 1 by 2 and add in row 3. 



Add -5/2 times row 2 to row 3, and add 3/2 times row 2 to row 4. 
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1 

0 

0 

0 


4 5 

2 4 

0 0 
0 0 


-9 -7 
-6 -6 
0 0 
-5 0 




R, 


- R', + Ra 


Interchange rows 3 and 4, we can produce a leading entry in column 4. 

,_Pivot 


' 1 

4 

5 

-9 

-7“ 


O 

* 

* 

* 


0 

2 

4 

-6 

-6 

General form 

0 

o 

=1= 

* 

* 

0 

0 

0 

-5- 

0 

0 

0 

0 

o 

* 

0 

L 

0 

0 

0 

0 _ 
Ph 

vot column 

0 

0 

0 

0 

0 


This is in echelon form and thus columns 1, 2, and 4 of .4 are pivot columns. 

positions 


columns 


,J 


(H -3 


1 


-6 4 

-1 -2-1-1 3 

-2 -3 0 3— 1 -1 

,1 4 5 -9 -7 


Pivot 
9 


Pivot 


Pivot element 


A pivot is a nonzero number in a pivot position that is used as needed to create zeros via 
row operations 

The Row Reduction Algorithm consists of four steps, and it produces a matrix in 
echelon form. A fifth step produces a matrix in reduced echelon form. 

The algorithm is explained by an example. 

Example 3 Apply elementary row operations to transform the following matrix first 
into echelon form and then into reduced echelon form. 

" 0 3 -6 6 4 -5" 

3 -7 8 -5 8 9 

3 -9 12 -9 6 15 
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Solution 


STEP 1 Begin with the leftmost nonzero column. This is a pivot column. The pivot 
position is at the top. 

" 0 3 -6 6 4 -5" 

3 -7 8 -5 8 9 

3 -9 12 -9 6 15 

1 Pivot column 


STEP 2 Select a nonzero entry in the pivot column as a pivot. If necessary, interchange 
rows to move this entry into the pivot position 

Interchange rows 1 and 3. (We could have interchanged rows 1 and 2 instead.) 


3- 

3 

0 


Pivot 

-9 12 -9 
-7 8 -5 

3-6 6 


6 

8 

4 


15 

9 

-5 


STEP 3 Use row replacement operations to create zeros in all positions below the pivot 


Subtract Row 1 from Row 2. i.e. 7?, — R x 

~ Pivot 

3 J -9 12 -9 6 

0 2 -4 4 2 

0 3 -6 6 4 


15 

-6 

-5 


STEP 4 Cover (or ignore) the row containing the pivot position and cover all rows, if 
any, above it. Apply steps 1 -3 to the sub-matrix, which remains. Repeat the process until 
there are no more nonzero rows to modify. 


With row 1 covered, step 1 shows that column 2 is the next pivot column; for step 2, 
we’ll select as a pivot the “top” entry in that column. 
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3 

0 

0 


rPivot 


-9 


12 


2 — 1 -4 

3 l 6 


-9 6 15 

4 2-6 

6 4-5 

Next pivot column 


According to step 3 “A// entries in a column below a leading entry are zero". For this 
subtract 3/2 time Rt from R 3 


3 

0 

0 


-9 12 -9 6 

2-442 
0 0 0 1 


15 

-6 

4 


R , — 

3 2 _ 


When we cover the row containing the second pivot position for step 4, we are left with a 
new sub matrix having only one row: 


3 -9 12 -9 6 15 

0 2 -4 4 2 -6 

0 0 0 0 In 4 


Pivot 


This is the Echelon form of the matrix. 

To change it in reduced echelon form we need to do one more step: 


STEP 5 Make the leading entry in each nonzero row 1. Make all other entries of that 
column to 0 . 

Divide first Row by 3 and 2 nd Row by 2 


1 -3 4 -3 2 5 

0 1-2 2 1-3 

0 0 0 0 1 4 




Multiply second row by 3 and then add in first row. 


1 0 -2 3 5 -4 

0 1-2 2 1-3 

0 0 0 0 1 4 


3R-, + 


Subtract row 3 from row 2, and multiply row 3 by 5 and then subtract it from first row 
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1 0 -2 3 0 -24 

0 1 -2 2 0 -7 

0 0 0 0 1 4 

This is the matrix is in reduced echelon form. 

Solutions of Linear Systems 

When this algorithm is applied to the augmented matrix of the system it gives solution set 
of linear system. 

Suppose, for example, that the augmented matrix of a linear system has been changed 
into the equivalent reduced echelon fonn 
[1 0-5 f 
0 114 
0 0 0 0 

There are three variables because the augmented matrix has four columns. The associated 
system of equations is 

Xj — 5 x 3 = 1 

x 2 + x 3 = 4 (1) 

0 = 0 which means x 3 is free 

The variables x / and x 2 corresponding to pivot columns in the above matrix are called 
basic variables. The other variable, X 3 is called a free variable. 

Whenever a system is consistent, the solution set can be described explicitly by solving 
the reduced system of equations for the basic variables in terms of the free variables. This 
operation is possible because the reduced echelon form places each basic variable in one 
and only one equation. 

In (4), we can solve the first equation for Xj and the second for x 2 . (The third equation is 
ignored; it offers no restriction on the variables.) 

Xj = 1 + 5 x 3 

x 2 = 4 - x 3 (2) 

x 3 is free 

By saying that X 3 is “free”, we mean that we are free to choose any value for X 3 . When 
X 3 = 0, the solution is (1, 4, 0); when X 3 = 1, the solution is ( 6 , 3, 1 etc). 

Note The solution in (2) is called a general solution of the system because it gives an 
explicit description of all solutions. 


R 2 -R 3 

R.-5R, 
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Example 4 Find the general solution of the linear system whose augmented matrix has 


been reduced to 


1 6 2 -5 -2 -4 

0 0 2 -8 -1 3 

0 0 0 0 1 7 


Solution The matrix is in echelon form, but we want the reduced echelon form 
before solving for the basic variables. The symbol before a matrix indicates that the 
matrix is row equivalent to the preceding matrix. 


1 6 2 -5 -2 -4 

0 0 2 -8 -1 3 

0 0 0 0 1 7 

By /?, + 2R 3 and R 2 + R, We get 


1 6 2 -5 0 10 
0 0 2 -8 0 10 
0 0 0 0 1 7 


By 



we get 


1 6 2 -5 0 10 
0 0 1 -4 0 5 
0 0 0 0 1 7 


By R ] - 2 R 2 we get 


1 6 0 3 0 0 
0 0 1 -4 0 5 
0 0 0 0 1 7 


The matrix is now in reduced echelon form. 

The associated system of linear equations now is 
Xj + 6x 2 + 3x 4 = 0 

x 3 - 4x 4 = 5 (6) 

x 5 =7 

The pivot columns of the matrix are 1,3 and 5, so the basic variables are Xj, X 3 , and X 5 . 
The remaining variables, X 2 and x 4 , must be free. 
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Solving for the basic variables, we obtain the general solution: 

r 


< 


xi = - 6 x 2 -3x 4 
x 2 is free 
x 3 = 5 + 4x 4 
x 4 is free 
X 5 = 7 


Note that the value of X5 is already fixed by the third equation in system (6). 

Exercise 

1. Find the general solution of the linear system whose augmented matrix is 


1-3-50 
0 113 


(7) 


2. Find the general solution of the system 

v, - 2x 2 - x 3 + 3x 4 = 0 
-2xj + 4 x 2 + 5x 3 - 5 x 4 = 3 
3xj - 6 x 2 - 6x 3 + 8x 4 = 2 


Find the general solutions of the systems whose augmented matrices are given in 
Exercises 3-12 



'1 

0 2 5" 



l 

0 

3. 

2 

0 3 6 


4. 

-3 7 0 


' 0 

3 6 

9 ' 


"1 3 -3 7" 

5. 

-1 

1 -2 

-1_ 

6. 

3 9 -4 1 


f 1 

2 -7' 



f \ 2 4 N 

7. 

-1 

-1 1 


8. 

-2 -3 -5 


v ^ 

1 5 

V 



v 7 1-1, 


-5 

9 
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' 2 



3 f 





fl 

0 

-9 

0 

4 N 

-4 

















0 

1 

3 

0 

-1 

-6 

12 


-9 



10. 


0 

0 

0 

1 

-7 

k 4 

-8 


6 1 










V 


j 





vO 

0 

0 

0 

1, 

f 1 

-2 

0 

0 

7 

-3> 


fl 

0 

-5 

0 

-8 

3^ 

0 

1 

0 

0 

-3 

1 

12. 

0 

1 

4 

-1 

0 

6 

0 

0 

0 

1 

5 

-4 


0 

0 

0 

0 

1 

0 

vO 

0 

0 

0 

0 

0; 


1° 

0 

0 

0 

0 



Determine the value(s) of h such that the matrix is the augmented matrix of a consistent 
linear system. 



' 1 4 2] 


"1 h 3" 

13. 

-3 h -1 

14. 

2 8 1 


Choose h and k such that the system has (a) no solution, (b) a unique solution, and (c) 
many solutions. Give separate answer for each part. 


15. xi + hx2 = 1 
2 x\ + 3x2 = k 


16. xi - 3x2 = 1 
2xi + hx2 = k 
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Lecture 5 

Vector Equations 

This lecture is devoted to connect equations involving vectors to ordinary systems of 
equations. The term vector appears in a variety of mathematical and physical contexts, 
which we will study later, while studying “Vector Spaces”. Until then, we will use vector 
to mean a list of numbers. This simple idea enables us to get interesting and important 
applications as quickly as possible. 

Column Vector 

“A matrix with only one column is called column vector or simply a vector ”. 



_ 


"2l 


e.g. U =[3 -if = 

3 

-1 

V = [2 3 5f = 

3 

, >V = ^w 1 w 2 w 3 w 4 ] are all 


_ — 


5 



column vectors or simply vectors. 

Vectors in R - 

If R is the set of all real numbers then the set of all vectors with two entries is denoted 


For example: the vector U = \3 -if = 


3 

-1 


Here real numbers are appeared as entries in the vectors, and the exponent 2 indicates that 
the vectors contain only two entries. 

Similarly if 3 and R 4 contain all vectors with three and four entries respectively. The 
entries of the vectors are always taken from the set of real numbers if. The entries in 
vectors are assumed to be the elements of a set, called a Field. It is denoted by F. 

Algebra of Vectors 

Equality of vectors in R~ 

Two vectors in R 2 are equal if and only if their corresponding entries are equal. 


■2j L y 2 


If u = 


V = 


then u =v iff 


A 



as 4 = 4 but 6^3 
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Note In fact, vectors 


in R 2 are nothing but ordered pairs of real numbers both 


representing the position of a point with respect to origin. 

Addition of Vectors 

Given two vectors u and v in R 2 , their sum is the vector u + v obtained by adding 
corresponding entries of the vectors u and v, which is again a vector in Hi 2 



lt ] 


v i 

_ 9 

u \ 


V 1 


u x + Vj 

For u — 

, v = 

e M~ Then u + v = 

+ 

1 

= 


U 2 


3 _ 


_« 2 _ 


3 _ 


_u 2 +v 2 _ 


For example, 


~ 1 “ 


“2 


1 + 2 


"3“ 


+ 


= 




_—2_ 


_5_ 


2 + 5_ 


_3_ 


Scalar Multiplication of a vector 


Given a vector u and a real number c, the scalar multiple of u by c is the vector cu 
obtained by multiplying each entry in u by c. 

' 3 ' 

-1 


For example, if u = 


and c = 5, then cu = 5 


" 3 ' 


"15" 

-1 


-5 


Notations The number c in cu is a scalar; it is written in lightface type to distinguish it 
from the boldface vector u. 

~ 2 


Example 1 Given u = 


1 

-2 


and v = 


-5 


, find 4 m, (-3) v, and 4u + (-3) v 


Solution 


"1 


"4x1 


' 4 ' 

-2 


4 x(-2) 


_- 8 _ 


(-3)v = (-3) 



And 


4 u + (-3)v = 


' 4 ' 


-6 


"-2" 


+ 


= 


_- 8 _ 


_! 5 J 


7 
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Note: Sometimes for our convenience, we write a column vector 


3 

-1 


in the fonn 


(3, -1). In this case, we use parentheses and a comma to distinguish the vector (3, -1) 
from the 1x2 row matrix [3 -1], written with brackets and no comma. 


Thus 


3 

-1 


*[3 -1] but 


3 

-1 


(3,-1) 


Geometric Descriptions of R 

Consider a rectangular coordinate system in the plane. Because each point in the plane is 
determined by an ordered pair of numbers, we can identify a geometric point (a, b) with 
a 


the column vector 
See Figure 1. 


. So we may regard R ' as the set of all points in the plane. 


X 2 



- w ^ 

•(2, 2) 


X] 

(-2,-1). 

.(3,-1) 


Figure 1 Vectors as points. 


Vectors in R - 

Vectors in R 3 are 3x1 column matrices with three entries. They are represented 
geometrically by points in a three-dimensional coordinate space, with arrows from the 
origin sometimes included for visual clarity. 

Vectors in R - 

If n is a positive integer, R" (read “r-n”) denotes the collection of all lists (or ordered 
n- tuples) of n real numbers, usually written as n x 1 column matrices, such as 

U = [«J u 2 •■•U n Y 

The vector whose all entries are zero is called the zero vector and is denoted by O. 
(The number of entries in O will be clear from the context.) 
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Algebraic Properties of R - 

For all u, v, w in if" and all scalars c and d\ 


(i) u + v = v + u (Commutative) 

(ii) (u + v) + w = u + (v + w) (Associative) 

(iii) u + 0 = 0 + u = u (Additive Identity) 

(iv) u + (—u) =( —u) + u = 0 (Additive Inverse) 

where -u denotes (-l)u 

(v) c(u + v)=cu+cv (Scalar Distribution over Vector Addition) 

(vi) ( c + d)u = cu + du (Vector Distribution over Scalar Addition) 

(vii) c(du) = (cd)u 

(viii) lu=u 

Linear Combinations Given vectors Vi, V2, •••, v p in if" and given scalars a, C2, .... c p 
the vector defined by 


y = C ] V ] + C 1 V 1 H-h c p v p 

is called a linear combination of Vi, ... ,v p using weights ci,..., c p . 

Property (ii) above pennits us to omit parenthesis when forming such a linear 
combination. The weights in a linear combination can be any real numbers, including 
zero. 

Example 


For Vj = 


~-ll 

, v 0 = 

'2 

1 

~ L 

1 


, if w = — Vj - —v 2 then we say that w is a linear combination of 


Vi and V 2 . 

Example As (3, 5,2) = 3(1, 0,0) + 5(0, 1,0) + 2(0, 0 , 1) 

(3, 5,2) = 3 ^ + 5 v 2 + 2 v 3 where Vj= (1, 0,0), v 2 = (0, 1 , 0) v 3 =(0, 0 , 1) 


So (3, 5,2) is a vector which is linear combination of V, , v 2 , V 3 


Example 5 Let a , = 

' 1 ' 

-2 

, a 2 

"2l 

5 

, and b = 

1 

r- tj- 

i _ 


-5 


6 


-3 
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Determine whether b can be generated (or written) as a linear combination of ai and «?• 
That is, determine whether weights Xi and X2 exist such that 

X] ai + X2 (12 = b ( 1 ) 


If the vector equation ( 1 ) has a solution, find it. 

Solution Use the definitions of scalar multiplication and vector addition to rewrite the 
vector equation 


' 1" 


"2l 


“ 7 “ 

-2 

+ X, 

5 

= 

4 

-5 


6 


-3 


ai a 2 b 



1 

J-T 

1 _ 


1 

K> 

X 

to 

_1 


l 

r- 

1 _ 


-2xj 

+ 

5x 2 

= 

4 


-5xj 


1 

VO 
_l 


1 

m 

1 

_ 1 


Xj + 2 x 2 


“ 7 “ 

-2xj + 5x, 

= 

4 

-5Xj + 6x 2 


-3 


Xj + 2x 2 = 7 
=> -2xj -(- 5 x 2 = 4 
-5xj + 6x, = -3 


( 2 ) 

(3) 


We solve this system by row reducing the augmented matrix of the system as follows: 

1 2 7" 

-2 5 4 

-5 6 -3 
Bv R-, + 2 R X : R 2 + 5 R X 

12 7 

0 9 18 

0 16 32 

v9y 


By x 


f 1 A 


V 16y 


R 
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1 2 7 
~ 0 1 2 
0 1 2 

By R 3 - R 2 ;R l - 2R 2 

"1 0 3" 

-012 
0 0 0 

The solution of (3) is xy = 3 and x^ = 2. Hence b is a linear combination of ay and a 2 , 
with weights xy = 3 and X 2 = 2. 

Spanning Set 

If Vi, . . . , v p are in R n , then the set of all linear combinations of Vi, . . . , v p is 
denoted by Span { Vi ,... ,v p } and is called the subset of R n spanned (or generated) by 
Vi, . . . , v p . That is, Span { Vi, ... , v p } is the collection of all vectors that can be 
written in the form of CyVy + C 2 V 2 + ... + c p v p , with c 1 , . . . , c p scalars. 

If we want to check whether a vector b is in Span {vi, ... , v p } then we will see whether 
the vector equation 

x jVj +X 2 V 2 +... + x p v p = b has a solution, or 
Equivalently, whether the linear system with augmented matrix [ Vi, ... , v p b] has a 
solution. 

Note 

(1) The set Span { Vi ,... ,v p ] contains every scalar multiple of Vy 

because cvy = cvy + OV 2 + .... + 0v p i.e every cv,- can be written as a linear 
combination of Vy,... , v p 

(2) Zero vector=0e Span{v^,v^,---v } as 0 can be written as the linear combination of 

V,,V,,• • • V that is 0 =0„v 1 +0 F v. + —t0„v here for the convenience it is mentioned 
viz n 

that 0 V is the vector(zero vector) while 0 F is zero scalar (weight of all Vj, v,, • • • v n ) and in 
particular not to make confusion that 0 V and 0 F are same! 


© Virtual University Of Pakistan 


51 





Let v be a nonzero vector in R 3 . Then Span {v} is the set of all linear combinations of v 
or in particular set of scalar multiples of v, and we visualize it as the set of points on the 
line in R 3 through v and 0. 


If u and v are nonzero vectors in R 3 , with v not a multiple of u, then Span {u, v} is the 
plane in R 3 that contains u, v and 0. In particular, Span {u, v} contains the line in R 3 
through u and 0 and the line through v and 0. 

1 1 r 5 ] r-3 

-2 ,a 2 = -13 ,andb= 8 

3 J L ~ 3 J l_ 1 

Then Span {aj, « 2 } is a plane through the origin in R 3 . Does b lie in that plane? 

Solution First we see the equation xiai + X2CI2 = b has a solution? 

To answer this, row-reduce the augmented matrix [«/ a 2 b\. 

"1 5-3" 

-2 -13 8 

3 -3 1 

By R 2 +2R i 

"1 5 -3" 

-0-32 
0 18 10 

By R, + 6R 2 

“1 5 -3“ 

~ 0 -3 2 

0 0-2 

Last row => Ox, = -2 which can not be true for any value of x 2 e R 
=> Given system has no solution 
b £ Span{a t M 2 \ and 

in geometrical meaning, vector b does not lie in the plane spanned by vectors 
<2, and Cl 2 
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Linear Combinations in Applications 

The final example shows how scalar multiples and linear combinations can arise when a 
quantity such as “cost” is broken down into several categories. The basic principle for the 
example concerns the cost of producing several units of an item when the cost per unit is 
known: 


[number] [ cos? 1 [total 
[ofunits] [perunit] [cos? 


Example 7 A Company manufactures two products. For one dollar’s worth of product 
B, the company spends $0.45 on materials, $0.25 on labor, and $0.15 on overhead. For 
one dollar’s worth of product C, the company spends $0.40 on materials, $0.30 on labor 
and $0.15 on overhead. 



".451 


'.40' 

Let6 = 

.25 

and c = 

.30 


.15 


.15 


, then 6 and c represent the “costs per dollar of income” 


for the two products. 


a) What economic interpretation can be given to the vector 1006? 

b) Suppose the company wishes to manufacture xi dollars worth of product B and x? 
dollars worth of product C. Give a vector that describes the various costs the 
company will have (for materials, labor and overhead). 

Solution 



".451 


"45" 

(a) We have 1006 = 100 

.25 

= 

25 


.15 


15 


The vector 1006 represents a list of the various costs for producing $100 worth of product 
B, namely, $45 for materials, $25 for labor, and $15 for overhead. 


(b) The costs of manufacturing xi dollars worth of B are given by the vector xy 6 and the 
costs of manufacturing X 2 dollars worth of C are given by Xic. Hence the total costs 
for both products are given by the vector xjb +X 2 C. 
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Vector Equation of a Line 

Let xo be a fixed point on the line and v be a nonzero vector that is parallel to the 
required line. Thus, if x is a variable point on the line through xo that is parallel to v, 
then the vector x - x 0 is a vector parallel to v as shown in fig below, 



(b) 



So by definition of parallel vectors x- x<> = tv for some scalar t. 

it is also called a parameter which varies from — oo to + oo . The variable point x traces 
out the line, so the line can be represented by the equation 

X-Xo = tV -(1) (-00 < t < +oo) 

This is a vector equation of the line through xo and parallel to v. 

In the special case, where x 0 = 0, the line passes through the origin, it simplifies to 

X = tV (-00 <t< +oo) 

Parametric Equations of a Line in R 2 

Let x = (x, y)e R 2 be a general point of the line through Xo = (xo, yo) e R 2 which is 
parallel to 
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v = (a, b)e R 2 , then eq. 1 takes the form 

(x, y) - (xo, yo) = t(a, b) (-GO < t < +co) 

=^> (x - xo, y - yo) = (to, tb) (-go <t< +go) 

=^> x = xo + at, y = yo + bt (-go <t < +oo) 

These are called parametric equations of the line in R 1 . 

Parametric Equations of a Line in R 3 

Similarly, if we let x = (x, y, z) e R 3 be a general point on the line through 
Xo = ( x 0 , yo, zo) e R that is parallel to v = (a, b, c) e R ', then again eq. 1 takes the form 
(. x , y, z) = ( xo , yo, zo) + t(a, b, c) (-go <t < +co) 
x = xo + at, y = yo + bt, z = zo + ct (-go <t < +go) 

These are the parametric equations of the linen? R 

Example 8 

(a) Find a vector equation and parametric equations of the line in R 2 that passes 
through the origin and is parallel to the vector v = (-2, 3). 

(b) Find a vector equation and parametric equations of the line in R 3 that passes 
through the point P 0 (l, 2, -3) and is parallel to the vector v = (4, -5, 1). 

(c) Use the vector equation obtained in part (b) to find two points on the line that are 
different from Po. 

Solution 


(a) We know that a vector equation of the line passing through origin is x = tv. 

Let x = (x, y). Then this equation can be expressed in component form as 
(x, y) = t (-2, 3) 

This is the vector equation of the line. 

Equating corresponding components on the two sides of this equation yields the 
parametric equations x = — 2t, v = 3/ 
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(b) The vector equation of the line is x = Xo + tv. 

Let x = (x, y, z), Here Xo = ( 1,2, -3) and v = (4, -5, 1), then above equation can 
be expressed in component form as 

(x,y,z) = ( 1,2, -3) + t (4, -5, 1) 

Equating corresponding components on the two sides of this equation yields the 
parametric equations 

x = 1 + At, y = 2 - 5t, z = —3 + t 

(c) Specific points on a line can be found by substituting numerical values for the 
parameter t. 

For example, if we take t = 0 in part (b), we obtain the point ( x, y, z) = (1, 2, -3), 
which is the given point P 0 . 

t = 1 yields the point (5,-3, -2) and 
t = -1 yields the point (-3, 7, - 4). 

Vector Equation of a Plane 

Let xo be a fixed point on the required plane W and Vi and V 2 be two nonzero vectors that 
are parallel to W and are not scalar multiples of one another. If x is any variable point in 
the plane W. Suppose v\ and V 2 have their initial points at xo, we can create a 
parallelogram with adjacent side’s t\V\ and t 2 v 2 in which x - Xo is the diagonal given by 
the sum 

= t\V 1 + t 2 v 2 


or, equivalently, x = Xo + t\V\ + t 2 v 2 -(1) 

where t\ and t 2 are parameters vary independently from -oo to +oo , 

This is a vector equation of the plane through .v 0 and parallel to the vectors Vi and v 2 . In 
the special case where xo = 0, then vector equation of the plane passes through the origin 
takes the form 
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X = t\V\+t 2 V 2 (—CO <t x < +CO, —GO <t 2 < +Go) 


Parametric Equations of a Plane 

Let x = (x, y, z) be a general or variable point in the plane passes through a fixed point 
jc 0 = (x 0 , Vo, z 0 ) and parallel to the vectors Vi = (a i, hi, ci) and v 2 = (<32, h 2 , c 2 ), then the 
component form of eq. 1 will be 

(x, y, z ) = (x 0 , vo, z 0 ) + L(ai, hi, ci) +t 2 (a 2 , h 2 , c 2 ) 

Equating corresponding components, we get 
x = xo +ci\t\ + a 2 t 2 

y = Vo + h 1 1 1 + h 2 t 2 (-00 < < +oo, -oo < t 2 < +oo) 

z = Zo + ciL + c 2 t 2 

These are called the parametric equations for this plane. 

Example 9 (Vector and Parametric Equations of Planes) 

(a) Find vector and parametric equations of the plane that passes through the origin of 
R 3 and is parallel to the vectors vi = (1, -2, 3) and v 2 = (4, 0, 5). 

(b) Find three points in the plane obtained in part (a). 

Solution 


(a) As vector equation of the plane passing through origin is x = t\V\ + t 2 v 2 . 
Let x = (x, y, z) then this equation can be expressed in component form as 
(x, y, z) = t\{ 1, -2, 3) + t 2 (4, 0, 5) 

This is the vector equation of the plane. 

Equating corresponding components, we get 
x = ti+4t 2 , y = -2ti, z = 3t\+5t 2 
These are the parametric equations of the plane. 
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(b) Points in the plane can be obtained by assigning some real values to the 
parameters t\ and t 2 : 

1 1=0 and t 2 = 0 produces the point (0, 0, 0) 
t\ = -2 and t 2 = 1 produces the point (2, 4, -1) 

1 1 = Vi and t 2 = Vi produces the point (5/2, -1, 4) 

Vector equation of Plane through Three Points 

If xo, V i and x 2 are three non collinear points in the required plane, then, obviously, the 
vectors v\ = jci - Xo and v 2 = x 2 - Xo are parallel to the plane. So, a vector equation of the 
plane is 

X = X 0 + U(Xi - Xo) + t 2 (x 2 - Xo) 

Example Find vector and parametric equations of the plane that passes through the 
points. P( 2, - 4, 5), Q (-1, 4, -3) and R( 1, 10, -7). 

Solution 

Let x = (x, y, z), and if we take Xo, Xi and X 2 to be the points P, Q and R respectively, 
then Xj — x 0 = PQ = (-3,8,-8) and x 2 -x 0 = PR = (-1,14,-12) 

So the component form will be 

(x, y, z ) = (2, -4,5) + t x (-3,8, -8) + t 2 (-l, 14, -12) 

This is the required vector equation of the plane. 

By equating corresponding components, we get 

x = 2-3 t x -t 2 , y = -4 + 8tj + 14t 2 , z = 5-8t 1 -12t 2 
These are the parametric equations of the required plane. 

Question: How can you tell that the points P, Q and R are not collinear? 

Finding a Vector Equation from Parametric Equations 

Example 11 Find a vector equation of the plane whose parametric equations are 

x = 4 + 5 t x —t 2 , y = 2- t x + St 2 , z = t x +t 2 
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Solution First we rewrite the three equations as the single vector equation 

( x,y,z ) = (4 + 5tj -t 2 ,2-t l +8 t 2 ,t 1 +t 2 ) 

=^> (x, y,z ) = (4,2,0) + (5f t + (~t 2 , 8 1 2 , t 2 ) 

=> (x, v, z) = (4,2,0) + t, (5,-1,1) + 1 2 (-1,8,1) 

This is a vector equation of the plane that passes through the point (4, 2, 0) and is parallel 
to the vectors vi = (5, -1, 1) and V 2 = (-1, 8, 1). 

Finding Parametric Equations from a General Equation 

Example 12 Find parametric equations of the plane x - y + 2z = 5. 

Solution First we solve the given equation for x in terms of y and z 
x = 5 + y - 2z 

Now make y and z into parameters, and then express x in terms of these parameters. 

Let y = t\ and z = t 2 

Then the parametric equations of the given plane are 

x=5 + t\-2t2, y = t\, z = t 2 


Exercises 

1. Prove that u + v = v + u for any u and v in R". 

2. For what value(s) of h, y belongs to Span {vy, v 2 , v^}? Where 


V 1 = 

' 1' 

-1 

> V 2 = 

“ 5 “ 

-4 

V 3 = 

"-3” 

1 

,and y = 

~-4~ 

3 


-2 


-7 


0 


h 


3. Detennine whether b is a linear combination of 07 , a 2 , and a 2 . 
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'll 


'-2] 


'-6l 


'll' 

a \ = 

0 

, a 2 — 

3 

,a 3 = 

7 

,b = 

-5 


1 


-2 


5 


9 



'1 


'-4l 


'2 ' 


"3 

ii). a l = 

0 

, u 2 

3 

,a 3 = 

5 

,b = 

-7 


-2 


8 


-4 


-3 


4. Determine if b is a linear combination of the vectors fonned from the columns of the 
matrix A 


A = 

' 1 0 2' 

-2 5 0 

,b = 

~-5~ 

11 

ii). A = 

" 1 0 5 1 

-2 1 -6 

,b = 

'2 ' 

-1 


2 5 8 


-7 


0 2 8 


6 


In exercises 7-10, list seven vectors in Span {vi, V 2 }. For each vector, show that the 
weights on V/ and V 2 used to generate the vector and list the three entries of the vector. 
Give also geometric description of the Span {vi, V 2 }. 



l5l 


f 1 > 


r-2> 


rr 

7. 

-1 

> V 2 = 

1 

8. Vj = 

0 

> V 2 = 

0 


13; 


1 - 5 ; 


1 1 J 


i^J 


9 . 

v i = 

(2 3 

6 

, v 2 = 

l-3l 

-9 



1-4; 


UJ 




^-3.7^ 


^5.8" 

10. 

v i = 

-0.4 

> V 2 = 

2.1 



v 11 *2 J 


15.3 J 
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11. Let a x = 

'1 

3 

, a 2 

1 

1 1 
00 fyi 
_ 1 

,b = 

1 

cn 1 

1 _ 


-1 


2 


h 


. For what value(s) of h is b in the plane spanned 


by ai and 02 ? 


12. Let Vj = 

1 

0 

> V 2 = 

-2 

1 

, and y = 

m 

^ 1 


-2 


7 


-5 


For what value(s) of h is y in the plane 


generated by v t and v 2 ? 



" 2 1 


"2l 


~h 

13. Let u = 

-1 

and 

1 

. Show that 

k 


is in Span{u, v} for all h and k. 
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Lecture 6 

Matrix Equations 

A fundamental idea in linear algebra is to view a linear combination of vectors as the 
product of a matrix and a vector. The following definition will permit us to rephrase some 
of the earlier concepts in new ways. 

Definition If A is an m x n matrix, with columns a/, a 2 , , a„ and if x is in R", then the 

product of A and x denoted by Ax, is the linear combination of the columns of A using 
the corresponding entries in x as weights, that is, 


Ax = [flj a 2 


x \ 



= Xjtfj + x 2 a 2 +... + x n a n 


Note that Ax is defined only if the number of columns of A equals the number of entries 
in*. 

Example 1 


4 


" 1 

2 -f 

1 

3 

= 4 

T 

+3 

' 2 ' 

+ 7 

'-f 


"4” 

+ 

" 6 ' 

+ 

"-7“ 


"3" 

0 

-5 3 _ 

H 


o_ 


_-5_ 


3 


0_ 


-!5_ 


21 


6 


7 


" 2 

-3" 



' 2 ' 


"-3” 


“ 8 " 


'- 21 ' 


-13 



"4l 











8 

0 

7 

= 4 

8 

+ 7 

0 

— 

32 

+ 

0 

— 

32 

-5 

2 



-5 


2 


-20 


14 


-6 


Example 2 For Vj, V 2 , V 3 in R"\ write the linear combination 3 Vi - 5 V 2 + 7 V 3 as a 
matrix times a vector. 

Solution Place Vi, V 2 , V 3 into the columns of a matrix ,4 and place the weights 3, -5, 
and 7 into a vector x. 


That is. 


3vj - 5 v 2 + 7v 3 = [Vj v 2 v 3 ] 


3 

-5 

7 


= Ax 
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We know how to write a system of linear equations as a vector equation involving a 
linear combination of vectors. For example, we kn ow that the system 


Xj + 2x 2 -x 3 =4 
- 5 x 2 + 3x 3 = 1 


is equivalent to 



V 


~ 2 1 




~4~ 

X, 


+ x~. 


+ x 3 


= 


1 

0 _ 

L 

_-5_ 

3 


1 


Writing the linear combination on the left side as a matrix times a vector, we get 


"1 

2 

-f 

Xj 


~4~ 

0 

-5 

3 _ 

X, 

— 

1 




_ X 3_ 




Which has the form Ax = b , and we shall call such an equation a matrix equation, to 
distinguish it from a vector equation. 


Theorem 1 If A is an m x n matrix, with columns «/, U 2 ,... , a n and if b is in if'", the 
matrix equation Ax = b has the same solution set as the vector equation 
xiai +X 2 (i 2 + ... +x„a„ = b 

which, in turn, has the same solution set as the system of linear equations whose 
augmented matrix is [<;/, a 2 ... a n b ] 

Existence of Solutions The equation Ax = b has a solution if and only if b is a linear 
combination of the columns of A 



" 1 3 4 ] 


b x 

Example 3 Let A = 

1 

l l 

On 

i__ 

and b = 

b 2 

]>3_ 


Is the equation Ac = b consistent for all possible bi, b 2 , bj? 


Solution Row reduce the augmented matrix for Ax = b : 


4R X + R 2 ,3 ifj + 


" 1 

3 

4 

b \ 


1 

3 

4 

b i 

-4 

2 

-6 

b 2 

~ 

0 

14 

10 

R + 4hj 

-3 

-2 

-7 

b 3 \ 


0 

7 

5 

b 3 + 3b { 
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r,--r 7 

3 2 

1 -2 -1 b , 

0 14 10 i> 2 +4i>, 

0 0 0 


The third entry in the augmented column is b 3 +3b x - —(Z> 2 + 4/t,) 

The equation Ax = b is not consistent for every b because some choices of b can make 


h x 


-—h 2 + b 3 nonzero. 


The entries in b must satisfy b x — dr /? 2 -+- Z? 3 = O 


This is the equation of a plane through the origin in R 3 . The plane is the set of all linear 
combinations of the three columns of A. See figure below. 



The equation Ax = b fails to be consistent for all b because the echelon form of A has a 
row of zeros. If A had a pivot in all three rows, we would not care about the calculations 
in the augmented column because in this case an echelon fonn of the augmented matrix 
could not have a row such as [0 0 0 1]. 
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Example 4 Which of the following are linear combinations of 


A = 

(a) 


4 0 

-2 -2 ’ 
"6 -8 
-1 -8 


(b) 


0 0 
0 0 




0 2 
1 4 


(c) 


6 0 
3 8 


Solution 


(a) 


6 

-1 



— aA + bB + cC 



o 

^t" 


"l 

-f 


"0 

2 

a 

-2 -2 

+ b 

2 

3 

+ c 

1 

4 


4 a + b -b + 2c 

-2a + 2 b + c -2a + 3b + 4c 


=> 4a + b = 6 (1) 

-b + 2c = -8 (2) 

-2a + 2b + c = -1 (3) 

-2a + 3b + 4c = -8 (4) 


Subtracting equation (4) from equation (3), we obtain 
-b - 3c = 7 (5) 

Subtracting equation (5) from equation (2): 

5c = -15 => c = -3 


From (2), -b + 2(-3) = -8 => b = 2 

From (3), -2a + 2(2) - 3 = -1 => a=l 

Now we check whether these values satisfy equation (1). 
4(1)+ 2 = 6 
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It means that 



-8 

-8 


is the linear combination of A, B and C. 


Thus 


6 -8 

-1 -8 


1A + 2B-3C 


(b) 


0 

0 


0 

0 


— aA + bB + cC 



1 

o 

1 _ 


"l 

-f 


"0 

2 " 

a 

-2 -2 

+ b 

2 

3 

+ c 

1 

4 


4 a + b -b + 2c 

-2a + 2 b + c -2a + 3b + 4c 


=> 

4a + b = 0 

( 1 ) 


-b + 2c = 0 

(2) 


-2a + 2b + c = 0 

( 3 ) 


-2a + 3b + 4c = 0 

( 4 ) 


Subtracting equation (3) from equation (4) we get 
b + 3c = 0 (5) 

Adding equation (2) and equation (5), we get 
5c = 0 => c = 0 


Put c = 0 in equation (5), we get b = 0 
Put b = c = 0 in equation (3), we get a = 0 
=> a = b = c =0 


It means that 


0 0 
0 0 


is the linear combination of A, B and C. 


Thus 


0 0 
0 0 


= 0A + OB + 0C 


(c) 


6 0 
3 8 


—aA+bB+cC 
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1 

o 

^t - 

1_ 


“1 

-f 


"0 

2" 

a 

-2 -2 

+ b 

2 

3 

+ c 

1 

4 


4 a + b -b + 2c 

-2a + 2 b + c -2a + 3b + 4c 

=> 4a + b = 6 (1) 

-b + 2c = 0 (2) 

-2a + 2b + c = 3 (3) 

-2a + 3b + 4c = 8 (4) 

Subtracting (4) from (3), we obtain 

-b - 3c = -5 (5) 

Subtracting (5) from (2): 

5c = 5 => c = 1 


From (2), -b + 2(1) = 0 => b = 2 


From (3), -2a + 2(2) + l= 3=>a=l 

Now we check whether these values satisfy (1). 
4(1)+ 2 = 6 


It means that 


6 

3 


0 

8 


is the linear combination of A, B and C. 


Thus 


6 0 
3 8 


1A + 2B+1C 


Theorem 2 Let A be an in x n matrix. Then the following statements are logically 
equivalent. 

(a) For each b in if'", the equation^ = b has a solution. 

(b) The columns of A Span if'". 

(c) A has a pivot position in every row. 
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This theorem is one of the most useful theorems. It is about a coefficient matrix, not an 
augmented matrix. If an augmented matrix [A b] has a pivot position in every row, then 
the equation Ax = b may or may not be consistent. 


Example 4 

Compute Ax, where A = 

“ 2 3 4 1 

-1 5 -3 

and x = 

Xj 

x 2 

Solution 

From the definition, 

6-2 8 




"2 

3 

4" 

*t 


" 2 1 


" 3 ' 


' 4 ' 

-1 

5 

-3 

x 2 

= Xi 

-1 

+ x 2 

5 

+ x 3 

-3 

6 

-2 

8 

x 3 


6 


-2 


8 


2 x, 


3 x 2 


1 

i_ 

-Xj 

+ 

5 x 2 

+ 

- 3 x 3 

6 xj 


1 

1 

K) 

1 _ 


1 

oo 

_ 1 


2xj + 3x 2 + 4x 3 
-Xj + 5 x 2 - 3 x 3 
6xj - 2x, + 8x 3 


Note 

In above example the first entry in Ax is a sum of products (sometimes called a dot 
product), using the first row of A and the entries in x. 


That is 


[2 3 4 ] 


[2xj + 3 x 2 + 4x 3 ] 


Examples 

In each part detennine whether the given vector span R 


(a) v i =(2, 2, 2), v 2 =(0, 0, 3), 
v 3 = (0, 1, 1) 

(b) Vj = (3, 1, 4), v 2 =(2,-3, 5), 
v 3 =(5,-2, 9), v 4 =(1, 4,-1) 

(c) Vj =(1,2,6), v 2 =(3,4,1), 
v 3 =(4,3,1), v 4 = (3, 3,1) 
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Solutions 

(a) We have to determine whether arbitrary vectors b = (b v b 2 ,b 3 ) in R 3 can be 
expressed as a linear combination b = k x v\ + k 2 v 2 + k 3 v 3 of the vectors v 1 ,v 2 , v 3 

Expressing this in tenns of components given by 
(b, ,b 2 ,b i ) = k l ( 2 , 2, 2 ) + k 2 (0,0,3) +(0,1,1) 

(bj, b 2 , /y) = ( 2k\ + Ok 2 + 0 k 2 , 2k^ + 0 k 2 + k 3 , 2k^ + 3 k 2 + k^) 

2k x + 0k 2 + 0k 2 = b t 
2k x +0 k 2 + k 2 = b 2 
2k x + 2k 2 + k 3 =b 3 

"2 0 0" 

A = 2 0 1 has a non zero determinant 

2 3 1 

Now 

det(zl) = -6 + 0 
Therefore v 1; v 2 ,v 3 span R ' 

(b) The set S{ v 1; v 2 ,v 3 ,v 4 }of vectors in R 3 spans V= R 3 if 

CjVj + c 2 v 2 + c 3 v 3 + c 4 v 4 = d x w x + d 2 w 2 + d 2 w 2 .(1) 

with 

w, =(1,0,0) 
w 2 =(0,1,0) 
w 3 =(0,0,1) 

With our vectors v,,v 2 ,v 3 ,v 4 equation (1) becomes 

c) (3,1,4) + c 2 (2, -3,5) + c 3 (5, -2,9) + c 4 (1,4, -1) = d x (1,, 0,0) + d 2 (0,1,0) + d 2 (0,0,1) 
Rearranging the left hand side yields 
3 Cj + 2 c~! + 5c 3 + lc 4 = lt/j + 0 d 2 + 0 d 2 
lq - 3 c 2 - 2c 3 + 4c 4 = 0d { +1 d 2 + 0 d 3 
4cj + 5 c 2 + 9c 3 - lc 4 = 0d x + 0 d 2 +1 d 2 
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3 2 5 1 1 0 0 

1 -3 -2 4 0 1 0 

4 5 9 -1 0 0 1 


1 0 2 1 0 0 - 

2 

0 11-10 1 - 

2 

0 0 0 0 1 -3 -2 

The reduce row echelon form 


5 3 

10 11 0 — — 



17 

17 


-4 

1 

0 11-1 

0 — 



17 

17 


-7 

n 

0 0 0 0 

1 — 



17 

17 


Corresponds to the system of equations 


5 3 

lCj + 1 c 3 + 1 c 4 — (— )g?2 (~)^3 

-4 1 

lc 2 + lc 3 -l 1 c 4 — ( )d 2 + 


( 2 ) 


0 = 1 d.+(-)d 2 +(-—)d, 

1 17 2 17 3 

So this system is inconsistent. The set S does not span the space V. 

Similarly Part C can be solved by the same way. 

Exercise 









" 3 ' 







' 1 

5 -2 

0 ' 


-2 




l. 

Let 

A = 

-3 

1 9 

-5 

,x = 

0 

,and 

b = 

9 




4 

-8 -1 

7 





0 




“ 


— 


-4 



— J 


It can be shown that Ax = b. Use this fact to exhibit b as a specific linear 
combination of the columns of A 
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2. Let A = 


2 5 

3 1 


,u = 


4 

-1 


, and v = 


-3 

5 


. Verify A(u + v) = Au +Av. 


3. Solve the equation Ax = b , with A = 


" 2 

4 

-6l 


~2 ~ 

0 

1 

3 

,b = 

5 

-3 

-5 

7 


-3 



'-5] 


" 3 

5 ' 

4. Let u = 

-3 

and A = 

1 

1 


-6 


-2 

-8 


. Is u belongs to the plane in R 3 spanned by the 


columns of A? Why or why not? 



"8l 


~4 

3 

5 ' 

5. Let u = 

2 

and A = 

0 

1 

-1 


3 


1 

2 

0 


. Is u in the subset of R 3 spanned by the columns of 


A? Why or why not? 


6 . Let A = 


-3 1 

6 -2 


and b = 




. Show that the equation Ax = b is not consistent for all 


possible b , and describe the set of all b for which Ax = b is consistent. 


7. How many rows of A - 


1 3-2-2 

0 1-15 

-1-21 7 

110-6 


contain pivot positions? 


In exercises 8 to 13 , explain how your calculations justify your answer, and mention an 
appropriate theorem. 

" 1 3-4' 

3 2-6 span R 3 ? 

-5 -1 8 


8 . Do the columns of the matrix A = 
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9. Do the columns of the matrix A = 


1 3-2-2 

0 1-15 

-1-21 7 

110-6 


span R 4 ? 


10. Do the columns of the matrix A = 


0 0 2 
0 -5 1 

4 6-3 


span R ? 


11. Do the columns of the matrix A = 


3 5 

1 1 

-2 -8 


span R ? 



' 1 “ 


"O' 


' 1 ' 


0 


1 


0 

V 1 = 

-1 

> V 2 = 

0 

> v 3 = 

0 


0 


-1 


-1 


"1 


'-f 


'3 

V 1 = 

0 

> V 2 = 

3 

> V 3 = 

-2 


-1 


7 


-2 


. Does {vi, V2, V3}span R 4 ? 


. Does { Vi, V 2 , V 3 } spanR 3 ? 


14. It can be shown that 

"4 1 2 ' 

-2 0 8 

"-f 

4 

— 

" 4 1 

18 

. Use this fact(and no row operations) 


3 5-6 

2 


5 




" 4 1 


' 4 ' 


"ll 


' 2 ' 

to find scalars Ci, C2, C3 such that 

18 

= c i 

-2 

+ c 2 

0 

+ C 3 

8 


5 


3 


5 


-6 
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'll 


'll 


15. Let u = 

8 

,v = 

3 

, and w = 

1 

. It can be shown that 2u - 5v - w = 0. Use this 


4 


1 


3 



fact(and no row operations) to solve the equation 


"3 

l" 



V 

8 

3 

Xi 

= 

1 

4 

1 

_x 2 _ 


3 


Determine if the columns of the matrix span R 4 . 



' 7 

2 

-5 

8 ' 


'12 

-7 

11 

-9 5 " 


-5 

-3 

4 

-9 


-9 

4 

-8 

7 -3 

16. 

6 

10 

-2 

7 

17. 

-6 

11 

-7 

3 -9 


-7 

9 

2 

15 


4 

-6 

10 

-5 12 
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Lecture 7 

Solution Sets of Linear Systems 


Solution Set 


A solution of a linear system is an assignment of values to the variables xi, X 2 ,... , x n such 
that each of the equations in the linear system is satisfied. The set of all possible solutions 
is called the Solution Set 

Homogeneous Linear System 


A system of linear equations is said to be homogeneous if it can be written in the form 
Ax = 0 , where A is an m x n matrix and 0 is the zero vector in R"'. 

Trivial Solution 

A homogeneous system Ax = 0 always has at least one solution, namely, x = 0 (the zero 
vector in R "). This zero solution is usually called the trivial solution of the homogeneous 
system. 

Nontrivial solution 

A solution of a linear system other than trivial is called its nontrivial solution. 

i.e the solution of a homogenous equation Ax = 0 such that x ^ 0 is called nontrivial 

solution, that is, a nonzero vector x that satisfies Ax = 0. 

Existence and Uniqueness Theorem 

The homogeneous equation Ax = 0 has a nontrivial solution if and only if the equation 
has at least one free variable. 

Example 1 Find the solution set of the following system 

3x, + 5x, - 4x 3 = 0 
3xj + 2x 2 - 4x 3 = 0 
6xj + x 2 - 8x, = 0 


Solution 



"3 

5 

-4] 




" 0 " 

Let ,4 = 

3 

2 

-4 

, x = 

x 2 

, b = 

0 


6 

1 

-8 


_x 3 J 


0 


The augmented matrix is 
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3 5-40 

3 2-40 

6 1-80 


For solution set, row reduce to reduced echelon fonn 


3 5-40 

0-300 
0-900 
3 5 -4 0 " 

0-300 
0 0 0 0 

4 

1 0 — 0 

3 

0-100 

0 0 0 0 

4 

1 0 — 0 

3 

0 10 0 

0 0 0 0 

4 

x, —x, = 0 

1 3 3 

x 2 =0 

0 =0 


— li?j + R ? , —2i?j + R 3 


- 3 R 2 + R 3 


1/3R 1 ,1/3R 2 ,5/3R 2 +R 1 




It is clear that X 3 is a free variable, so Ax = 0 has nontrivial solutions (one for each 
choice of X 3 ). From above equations we have, 


*t 



x 2 = 0 , with X 3 free. 


As a vector, the general solution of Ax = 0 is given by: 
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'4 


"4l 


~4~ 

X = 

_1 

_ 

3* 3 

0 

= x 3 

3 

0 

= x 3 v, where v = 

3 

0 


Uj 


x 3 


1 


1 


This shows that every solution of Ax = 0 in this case is a scalar multiple of v (it means 
that v generate or spans the whole general solution).The trivial solution is obtained by 
choosings = 0 . 

Geometric Interpretation 

Geometrically, the solution set is a line through 0 in R 3 , as given in the Figure below: 



Note: A nontrivial solution x can have some zero entries so long as not all of its entries 
are zero. 

Example 2 

Solve the following system 

lCbq-3x 2 -2x 3 = 0 (1) 


Solution 

Solving for the basic variable Xj in terms of the free variables, 
dividing eq. 1 by 10 and solve for x 

Xj = 0.3x2 + 0 .2x3 where X 2 and *3 free variables. 


As a vector, the general solution is: 





0.3x 2 +0.2x 3 


0.3x 2 


0.2x 3 

x = 

x 2 

= 

x 2 

= 

x 2 

+ 

0 


x 3 


x 3 


0 


x 3 
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"0.3" 


"0.2" 

x 2 

1 

+ x 3 

0 


0 


1 






U V 


( 2 ) 


This calculation shows that every solution of (1) is a linear combination of the vector u, v 
shown in (2). That is, the solution set is Span {u, v} 

Geometric Interpretation 

Since neither u nor v is a scalar multiple of the other, so these are not parallel, the 
solution set is a plane through the origin, see the Figure below: 


X3 



Note: 

Above examples illustrate the fact that the solution set of a homogeneous equation 
Ax = Ocan be expressed explicitly as Span {vi, V 2 , ..., v^} for suitable vectors 
v i, V 2 , ... , ^(because solution sets can be written in the form of linear combination of 
these vectors). If the only solution is the zero-vector then the solution set is Span {0}. 

Example 3 (For Practice) Find the solution set of the following homogenous system: 

Xj + 3 x 2 + x 3 = 0 

-4x i - 9x 2 + 2x 3 = 0 
- 3x 2 - 6x 3 = 0 

Solution: 



r 1 

3 

1 “ 




“0“ 

Let A = 

-4 

-9 

2 

, x = 

x 2 

, b = 

0 


0 

-3 

-6 


_X 3 J 


0 


The augmented matrix is: 
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1 

-4 

0 

1 

0 

0 


1 

0 

0 

1 

0 

0 


so 


x l 


3 1 

-9 2 

-3 -6 

3 1 0 

3 6 0 

-3 -6 0 

3 1 0" 

3 6 0 

0 0 0 
0-5 0 

1 2 0 
0 0 0 

- 5x 3 = 0 
x 2 + 2x 3 = 0 

0 = 0 


0 

0 

0 


4 , 


R 2 + R 2 

— R 2 , (-3)R 2 + R x 


From above results, it is clear that X 3 is a free variable, so Ax = 0 has nontrivial solutions 
(one for each choice ofx^). 

From above equations we have, 

Xj = 5x 3 , x 2 = -2x 3 , with X 3 a free variable. 

As a vector, the general solution of Ax = 0 is given by 


X = 

1_ 

— 

5x 3 

-2x 3 

= x 3 

1 

1 

K> fyi 
_ 1 

= x 3 v, where v = 

' 5“ 

-2 


X 3 


x 3 


1 


1 


Parametric Vector Form of the solution 


Whenever a solution set is described explicitly with vectors, we say that the solution is in 

parametric vector form 

The equation 

x = su + tv (s, t in R ) 

is called a parametric vector equation of the plane. It is written in this fonn to 
emphasize that the parameters vary over all real numbers. 

Similarly, the equation x = x ; v (with x 3 free), or x = tv (with t in R), is a parametric 
vector equation of a line. 
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Solutions of Non-homogeneous Systems 

When a non-homogeneous linear system has many solutions, the general solution can be 
written in parametric vector form as one vector plus an arbitrary linear combination of 
vectors that satisfy the corresponding homogeneous system. 

To clear this concept consider the following examples, 


Example: 5 Describe all solutions of Ax = b, where 


A = 

' 3 

-3 

5 

-2 

-4 

4 

and b = 

' 7 “ 

-1 


6 

1 

-8 


-4 


Solution 

Row operations on [A b] produce 


3 5-47 

-3 -2 4 -1 

6 1-8-4 

" 3 5 -4 7 " 

- 0 3 0 6 

0-90 -18 

" 3 5 -4 7 ' 

- 0 1 0 2 

0 0 0 0 

'l 0 -* -ll 

3 

- 0 1 0 2 

0 0 0 0 

4 . 

x, —x, = -1 

1 3 3 

OR x 2 =2 

0 =0 
4 

Thus jCj =-l +—x 3 , x 2 =2, and jcj is free. 

As a vector, the general solution of Ax = b has the form 


7?j + R -,, — 2R l + R 2 

3R 2 + R 3 , -R 2 


— 5 R-, + , 
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*1 


-'V 3 


"-ll 


1 

OJ I 

& 

_1 


“-1 

X = 

x 2 

= 

2 

= 

2 

+ 

0 

= 

2 


x 3 


x 3 

i 


0 


x 3 


0 












P 


+ x 3 


4 

3 

0 

1 


v 


The equation x = p + X3V, or, writing t as a general parameter, 

x =p + tv (t in R) (3) 


Note 

We know that the solution set of this question when Ax = 0 (example 1) has the 
parametric vector equation 


x = tv(t in R) (4) 

With the same v that appears in equation (3) in above example. 

Thus the solutions of Ax = b are obtained by adding the vector p to the solutions of 
Ax = 0. The vector p itself is just one particular solution of Ax = b (corresponding to t = 0 
in (3)). 

The following theorem gives the precise statement. 

Theorem 


Suppose the equation Ax = b is consistent for some given b, and let p be a solution. 
Then the solution set of Ax = b is the set of all vectors of the form 
w =p + Vh, where V/, is any solution of the homogeneous equation Ax = 0. 

Example 6: (For practice) 


x l + 3x 2 + x 3 = 1 
-4xj -9x, +2x 3 = -1 
- 3x 2 - 6x 3 = -3 

Solution 


Let A = 

r 1 3 

-4 -9 

1 “ 

2 

, x = 

X x 

x 2 

, b = 

f 

-1 


0 -3 

-6 


_X 3 J 


-3 


The augmented matrix is 
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13 11 

-4 -9 2 -1 

0 -3 -6 -3 

13 11 “ 

0 3 6 3 

0 -3 -6 -3 

13 11 " 

0 3 6 3 

0 0 0 0 
13 11 " 

0 12 1 
0 0 0 0 
1 0 -5 -2" 

0 12 1 

0 0 0 0 

Xj -5x 3 = -2 
x 2 + 2x 3 = 1 

0 = 0 


4i?j + R-,, 


R 2 + R, 



(-3 )R 2 +R, 


Thus Xj = -2 + 5x 3 , x 2 = 1 - 2x 3 , and x? is free. 

As a vector, the general solution of Ax = b has the form 



*t 


-2 + 5x 3 


~-2 


1 

& 

_i 


"-2“ 


“ 5“ 

X = 

x 9 

= 

1 - 2x 3 

= 

1 

+ 

-2 

= 

1 

+ x 3 

-2 


x 3 


x 3 


0 


x 3 J 


0 


1 


P v 


So we can write solution set in parametric vector form as 
x = p + x 3 v 
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Steps of Writing a Solution Set (of a Consistent System) 
in a Parametric Vector Form 

Step 1: 

Row reduces the augmented matrix to reduced echelon form. 

Step 2: 

Express each basic variable in terms of any free variables appearing in an 
equation. 

Step 3: 

Write a typical solution x as a vector whose entries depend on the free variables 
if any. 

Step 4: 

Decompose x into a linear combination of vectors (with numeric entries) using 
the free variables as parameters. 


Exercise 


Determine if the system has a nontrivial solution. Try to use as few row operations as 
possible. 

1. xi - 5 x 2 + 9x 3 = 0 
-Xi + 4x2 - 3x3 = 0 
2xi - 8x 2 + 9x 3 = 0 

3. 5xi - X 2 + 3x 3 = 0 
4xi - 3x2 + 7 x 3 = 0 

Write the solution set of the given homogeneous system in parametric vector form. 

4. xi - 3x2 - 2 x 3 = 0 5. xi + 2x2 - 7x 3 = 0 

X 2 -x 3 = 0 -2xi - 3x2 + 9x 3 = 0 

-2xi + 3x2 + 7 x 3 = 0 -2x2 + 10x 3 = 0 

In exercises 6-8, describe all solutions of Ax = 0 in parametric vector form where A is 
row equivalent to the matrix shown. 



'1 

-5 

0 

2 

0 

-4] 


'1 

6 

0 

8 

-1 

-2 


0 

0 

0 

1 

0 

-3 


0 

0 

1 

-3 

4 

6 

6. 

0 

0 

0 

0 

1 

5 

7. 

0 

0 

0 

0 

0 

1 


0 

0 

0 

0 

0 

0 


0 

0 

0 

0 

0 

0 


8. [1 -5 0 4] 


2. 3xj + 6 x 2 - 4x 3 - X 4 = 0 
-5xi + 8 x 3 + 3 x 4 = 0 

8xi-x 2 + 7x 4 = 0 
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9. Describe the solution set in R of xi - 4x2 + 3x3 = 0, compare it with the solution set 
of xi - 4x2 + 3x3 = 7. 

10. Find the parametric equation of the line through a parallel to b. 



11. Find a parametric equation of the line M through p and q. 



5 10 

12. Given A= -8 -16 , find one nontrivial solution of Ax = 0 by inspection. 

7 14 

"1 3" 

13. Given A= 2 6 , find one nontrivial solution of Ax = 0 by inspection. 

3 9 
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Lecture 8 

Linear Independence 


Definition 

An indexed set of vectors {vy, v 2 , ... , v p } in R" is said to be linearly 
independent if the vector equation +x 2 v 2 -\— + x v = 0 has only the trivial solution. 

The set {vy, v 2 ,..., v p } is said to be linearly dependent if there exist weights Cy, ...., c p , 
not all zero, such that qv l + c 2 v 2 H-b c v =0 (1) 

Equation (1) is called a linear dependence relation among Vy v p , when the weights 
are not all zero. 


Example 1 



'll 


"4l 


'2' 

Let Vj = 

2 

> V 2 = 

5 

> V 3 = 

1 


3 


6 


0 


(a) Determine whether the set of vectors {vy, v 2 , Vy} is linearly independent or not. 

(b) If possible, find a linear dependence relation among vi,v 2 ,v 2 . 

Solution 


(a) Row operations on the associated augmented matrix show that 


14 2 0 

2 5 10 

3 6 0 0 

14 2 0 

0-3-3 0 

0 - 6-6 0 

1 4 2 0 ' 

0-3-3 0 

0 0 0 0 


R 2 + R, 


(~2)R 1 + R 2 , (-3)Rj + R 3 


( 2 ) 


Clearly, x 2 and x 2 are basic variables and X 3 is free. Each nonzero value of x 2 detennines 
a nontrivial solution. 

Hence Vy, v 2 , Vy are linearly dependent (and not linearly independent). 

(b) To find a linear dependence relation among vy, v 2 , Vy, completely row reduce the 
augmented matrix and write the new system: 
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1 4 2 
0 1 1 
0 0 0 
1 0 
0 1 
0 0 


0 

0 

0 

-2 

1 

0 



0 

0 

0 


R t -4R 2 


x i 

=> x 2 


- 2x 3 = 0 
+ x 3 = 0 

0 =0 


Thus Xi= 2 x 3 , X 2 =-X 3 , and X 3 is free. 

Choose any nonzero value for * 3 , say, X 3 = 5, then Xi =10, and *2 = -5. 

Substitute these values into XjVj + x 2 v 2 + x 3 v 3 = 0 

=> 10vi - 5 v2 + 5v3 = 0 

This is one (out of infinitely many) possible linear dependence relation among Vi, v?, V 3 . 

Example (for practice) 

Check whether the vectors are linearly dependent or linearly independent. 

vj=(3,-l) v 2 = (-2,2) 

Solution 

Consider two constants Q andC 2 . Suppose: 

(3j - 1 ) + c 2 (- 2 , 2 ) = 0 
(3cj - 2 c 2 , + 2c 2 ) = (0,0) 

Now, set each of the components equal to zero to arrive at the following system of 
equations: 


3q - 2 c 2 = 0 
-Cj + 2 c 2 = 0 

Solving this system gives the following solution, 

Cj = 0 c 2 = 0 

The trivial solution is the only solution, so these two vectors are linearly independent. 
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Linear Independence of Matrix Columns 

Suppose that we begin with a matrix A = [a, ... a„] instead of a set of vectors. The 

matrix equation Ax = 0 can be written as +x 2 a 2 ■+-f- x n a n = 0 


Each linear dependence relation among the columns of A corresponds to a nontrivial 
solution of Ax = 0. 


Thus we have the following important fact. 


The columns of a matrix A are linearly independent if and only if the equation 
Ax = 0 has on jy the trivial solution. 


Example 2 


Determine whether the columns of A = 


0 

1 

5 


1 

2 

8 


4 

-1 

0 


are linearly 


independent. 

Solution To study Ax = 0, row reduce the augmented matrix: 


0 14 0 

12-10 
5 8 0 0 

1 2 -1 O' 
0 14 0 

5 8 0 0 

1 2 -1 0 
0 14 0 

0-250 
1 2 -1 O' 
0 14 0 

0 0 13 0 


R 


12 


(-5)R t +R 3 


(2)R 2 +R 2 


At this point, it is clear that there are three basic variables and no free variables. So the 
equation Ax = 0 has only the trivial solution, and the columns of A are linearly 
independent. 

Sets of One or Two Vectors 


A set containing only one vector (say, v) is linearly 
independent if and only if v is not the zero vector. This is because the vector equation 
XiV = 0 has only the trivial solution when v v 0. The zero vector is linearly dependent 
because XjO = 0 has many nontrivial solutions. 
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Example 3 

Check the following sets for linearly independence and dependence. 




"3" 


"6" 

a. 

A = 

1 

> V 2 = 

2 

b. 


"3" 


"6" 

A = 

2 

> V 2 = 

2 


Solution 


a) Notice that v 2 is a multiple of Vy, namely, v 2 = 2vi. 

Hence -2vi +v 2 = 0, which shows that {vi, v 2 } is linearly dependent. 

b) Vi and v 2 are certainly not multiples of one another. Could they be linearly 
dependent? 

Suppose c and d satisfy cv 2 + dv 2 = 0 

If c * 0, then we can solve for vy in terms of v 2 , namely, Vy = (-d/c) v 2 . This result is 
impossible because Vy is not a multiple of v 2 . So, c must be zero. Similarly, d must 
also be zero. 

Thus {vi, v 2 } is a linearly independent set. 


Note A set of two vectors {vi, v 2 } is linearly dependent if and only if one of the vectors 
is a multiple of the other. 

In geometric terms, two vectors are linearly dependent if and only if they lie on the same 
line through the origin. Figure 1, shows the vectors from Example 3. 

*2 


( 6 , 2 ) 


Xi 



Linearly dependent 
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Figure 1 Linearly independent 

Sets of Two or More Vectors 

Theorem (Characterization of Linearly dependent Sets) 

An indexed set s = {v 15 v 2 ,---,v } of two or more vectors is linearly dependent if and only 
if at least one of the vectors in S is a linear combination of the others. In fact, if S is 
linearly dependent, andv^O, then some v , (with j >1) is a linear combination of the 

preceding vectors, , • • •, v. j. 

Proof 

If some v in S equals a linear combination of the other vectors, then 
v . can be subtracted from both sides of the equation, producing a linear dependence 
relation with a nonzero weight (-1) on v.. 

For instance, if Vi = C 2 V 2 + C 3 V 3 , then 0 = (-l)vi + C 2 V 2 +C 3 V 3 + OV 4 + ... + 0v p . 

Thus S is linearly dependent. 

Conversely, suppose S is linearly dependent. If V/ is zero, then it is a (trivial) linear 
combination of the other vectors in S. 

If v^O and there exist weights Ci, ... , c p , not all zero(because vectors are linearly 
dependent), such that 

C 1 V 1 + C 2 V 2 +...+ c p v p = 0 

Let j be the largest subscript for which c. ^ 0 . If j = 1, then CiVi = 0, which is 
impossible because v . ^ 0 . 

So /' > 1, and CjV x + —f c / v / + 0v /+] +—i-0v /) =0 


CjVj = -C,v, - c 2 v 2 - C j-\Vj-x 


v ,= 


f \ 


V c u 


Vj + 


f \ 


V C JJ 


V 2 H-+ 


f \ 
C J -1 


V C t J 


v y-i 


Note: This theorem does not say that every vector in a linearly dependent set is a linear 
combination of the preceding vectors. A vector in a linearly dependent set may fail to be 
a linear combination of the other vectors. 
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3 1 

1 and v = 6 . Describe the set spanned by u and v, and prove 

°J L° 

that a vector w is in Span {u, v} if and only if {u, v, w} is linearly dependent. 

Solution 

The vectors u and v are linearly independent because neither vector is a multiple 
of the other, nor so they span a plane in R 3 . In fact, Span {u, v} is the Jt 7 Jt 2 -plane 
(with X 3 = 0). If w is a linear combination of u and v, then {u, v, w] is linearly dependent. 

Conversely, suppose that {u, v, w} is linearly dependent. 

Some vector in {u, v, w} is a linear combination of the preceding vectors (since u ^0). 
That vector must be w, since v is not a multiple of u. So w is in Span {u, v} 


*2 


XI w 

Linearly dependent w in Span {u, v}. 



Linearly independent w not in Span {u, v} 
Figure 2: Linear dependence in R 3 . 


This example generalizes to any set {u, v, w] in R 3 with u and v linearly independent. The 
set {u, v, w} will be linearly dependent if and only if w is in the plane spanned by u and v. 

Theorem 

If a set contains more vectors than there are entries in each vector, then the set 
is linearly dependent. That is, any set {v/, v?, ..., v p } in R" is linearly dependent if p > n. 
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Example 5 The vectors 


"2l 


" 4 1 


-2 

1 


-1 


2 


are linearly dependent, because there are 


three vectors in the set and there are only two entries in each vector. 


Notice, however, that none of the vectors is a multiple of one of the other vectors. See 
Figure 4. 


*2 



Theorem 


If a set S = {vi, v 2 , ... , v p \ in R" contains the zero vector, then the set is linearly 
dependent. 

Proof 

By renumbering the vectors, we may suppose that Vi = 0. 

Then (l)v 7 + 0v 2 + ... + 0v p = 0 shows that S is linearly dependent because in this 
relation coefficient of Vi is nonzero). 

Example 6 Determine by inspection if the given set is linearly dependent. 



“f 


“2“ 


"3l 


“4“ 


“2“ 


“O' 


T 

a. 

7 


0 


1 


1 

b. 

3 


0 


l 


6 


9 


5 


8 


5 


0 


8 


Solution 


“-2l 


“ 3 “ 

4 


-6 

6 

? 

-9 

iO 


!5 


a) The set contains four vectors that each has only three entries. So,the set is linearly 
dependent by the Theorem above. 

b) The same theorem does not apply here because the number of vectors does not exceed 
the number of entries in each vector. Since the zero vector is in the set, the set is 
linearly dependent by the next theorem. 
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c) As we compare corresponding entries of the two vectors, the second vector seems to 
be -3/2 times the first vector. This relation holds for the first three pairs of entries, but 
fails for the fourth pair. Thus neither of the vectors is a multiple of the other, and 
hence they are linearly independent. 


Exercise 


" 3 ' 


1 

kO 

1 

1_ 


" 0 ' 


" 3 " 

2 

,v = 

1 

,w = 

-5 

, and z = 

7 

1 

1 

1 _ 


7 


2 


1 

1 

1_ 


(i) Are the sets {u, v}, {u, w}, {u, z}, {v, w}, {v, z}, and {tv, z\ each linearly 
independent? Why or why not? 

(ii) Does the answer to Problem (i) imply that {u, v, w, z) is linearly independent? 

(iii) To detennine if {u, v, w, z) is linearly dependent, is it wise to check if, say w is a 
linear combination of u, v and z? 

(iv) Is {u, v, w, z\ linear dependent? 

Decide if the vectors are linearly independent. Give a reason for each answer. 



" 3 l 


“- 3 “ 


6 ] 


“l 


“- 3 “ 


“0 ' 

2 . 

0 


2 


4 

3 . 

3 


-5 


5 


0 


3 


0 


-2 


6 


-6 


Determine if the columns of the given matrix fonn a linearly dependent set. 







" 3 

4 

3 " 

" 1 

3 

-2 

0 " 


-1 

-7 

7 

3 

10 

-7 

1 

5 . 

1 

3 

-2 

-5 

-5 

3 

7 










0 

2 

-6 

" 1 

1 

0 

4 " 


' 1 

-1 

-3 

-1 

0 

3 

-1 

7 . 

0 

1 

5 

0 

-2 

1 

1 


-1 

2 

8 

1 

0 

-1 

3 


_ 3 

-1 

1 


For what values of h is V3 in span {vi, V2} and for what values of h is {vi, V2, V3} 
linearly dependent? 



1 


-2 


1 

8 . Vj = 

3 

» V 2 = 

-6 

> V 3 = 

2 


-2 


4 


h 



1 


3 


-2 

9 - v 1 = 

3 

a 2 = 

9 

> V 3 = 

-6 


3 


-1 


h 
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Find the value(s) of h for which the vectors are linearly dependent. 



Determine by inspection whether the vectors are linearly independent. Give reasons for 
your answers. 



"6 1 [3 
14. 2 ,1 

- 8 j [-2 


15. Given A = 


2 

-5 

-3 

1 


3 

1 

-1 

0 


5 

-4 

, observe that the third column is the sum of the first two 
-4 

1 


columns. Find a nontrivial solution of Ax = 0 without perfonning row operations. 


Each statement in exercises 16-18 is either true(in all cases) or false(for at least one 
example). If false, construct a specific example to show that the statement is not always 
true. If true, give a justification. 

16 . If vi, ..., V4 are in R 4 , and V3 = 2 vi + V2, then {vi, V2, V3, V4} is linearly dependent. 

17. If vi and V 2 are in R 4 , and vi is not a scalar multiple of V 2 , then {vi, V 2 } is linearly 
independent. 

18. . If vi, ..., V4 are in R 4 , andjvi, V2, V3} is linearly dependent, then {vi, V2, V3, V4} is 
also linearly dependent. 


" 8 -3 0 -7 2 " 

-9 4 5 11 -7 

19. Use as many columns of A = as possible to construct a 

J 6-22-44 

5 -17 0 10 

matrix B with the property that equation Bx = 0 has only the trivial solution. Solve 
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Bx = 0 to verify your work. 
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Lecture 9 

Linear Transformations 


Outlines 


• Matrix Equation 

• Transformation, Examples, Matrix as Transformations 

• Linear Transformation, Examples, Some Properties 

Matrix Equation 

An equation Ax = b is called a matrix equation in which a matrix A acts on a vector x by 
multiplication to produce a new vector called b. 

For instance, the equations 


4 

2 


-3 1 
0 5 


A 


l 

l 

l 

l 

'f 


X 


5 

8 


▲ 


b 


and 



' 1 ” 



4 -3 1 3“ 

4 


" 0 " 

2051 

-1 


0 


_ 3 _ 



A 

u 


o 


i V 


t 


A 


Solution of Matrix Equation 


Solution of the Ax = b consists of those vectors x in the domain that are transformed into 
the vector b in range. 

Matrix equation Ax = b is an important example of transformation we would see later in 
the lecture. 


Transformation or Function or Mapping 
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A transformation (or function or mapping) T from R " to R is a rule that assigns to 
each vector x in R", an image vector T(x) in R" 1 . 

T : R’ > R" 

The set R " is called the domain of T, and R m is called the co-domain of T. For x in R" 
the set of all images T(x) is called the range of T. 

Note 

To define a mapping or function, domain and co-domain are the ordinary sets. 
However to define a linear transfonnation, the domain and co-domain has to be 
M m (or M' ! ). Moreover a map T :IR m —>R”is a linear transformation if for any two 
vectors say u,v e M m and the scalars c x ,c 2 , the following equation is satisfied 
T (q u + c 2 v) = c t T (u) + c 2 T (v) 

Example 1 Consider a mapping T : R 2 —» R 2 defined by T(x,y) = (-x,y). This 
transfonnation is a reflection about y-axis in xy plane. 

Here 7(1,2) = (-1,2) . T has transformed vector (1,2) into another vector (-1,2). 

In matrix form: 



Further the projection transformation T :R 2 —»f? 2 defined by T(x,y) = (x, 0) is given as: 
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and define a transformation T :R 2 —» R 3 by T(x) =Ax, so that 



' 1 

-3] 

T (x) = Ax = 

3 

5 


-1 

7 r 


Xj - 3 x 2 
3xj + 5x 2 
-Xj +7x 2 


a) Find T (u), the image of u under the transformation T. 

b) Find an x in R 2 , whose image under T is b. 

c) Is there more than one x whose image under T is bl 

d) Determine if c is in the range of the transfonnation T. 


Solution (a) 



" 1 -3" 



“ 5 “ 



“2“ 



Au = 

3 5 

-1 

— 

1 


-1 7 

— 


-9 


Here T(u) = 


5 

1 

-9 


Here the matrix transfonnation has transformed u = 




“ 5 “ 

" 2 1 

into another vector 

1 

-1 



— 


-9 


(b) We have to find an x such that T (x) = b or Ax = b 


l. e 


" 1 -3" 



" 3 " 

3 5 


= 

2 


x 1 \ 



-1 7 



-5 


( 1 ) 


Now row reduced augmented matrix will be: 
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1 

3 

-1 

1 

0 

0 

1 

0 

0 

1 

0 

0 


-3 3 

5 2 

7 -5 

-3 3 ' 

14 -7 
4 -2 

-3 3 ' 

1 -.5 

0 0 
0 1.5" 

1 -.5 
0 0 


— 3i?j + + f? 3 


—, —4 R-, + R^, 
14 2 2 3 

3 R-, + 


Hence Xj = 1.5, x 2 = - 0.5, and x = ^ . 

The image of this x under T is the given vector b. 


(c) From (2) it is clear that equation 

(1) has a unique solution. So, there is exactly one x whose image is b. 


(d) The vector c is in the range of T if c is the image of some x in if 2 , that is, if 

c = T(x ) for some x. This is just another way of asking if the system Ax = c is 
consistent. To find the answer, we will row reduce the augmented matrix: 


1 

3 

-1 

1 

0 

0 

1 

0 

0 

1 

0 

0 


-3 3 
5 2 

7 5 

-3 3 " 

14 -7 
4 8 

-3 3 " 

1 2 
14 -7 
-3 3 

1 2 
0 -35 


— 3ifj + Rj , ifj + ifj 


4 if 3 , R 2 3 


-14i? 2 +i? 3 
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x, -3x 2 =3 
Ox, + x, = 2 

Ox, + 0x 2 = -35 => 0 = 35 but 0 ^ 35 

Hence the system is inconsistent. So c is not in the range of T. 


So from the above example we can view a transformation in the form of a matrix. 
We’ll see that a transformation T :R" —» R m can be transformed into a matrix of 
order m x n and every matrix of order m x n can be viewed as a linear 
transformation. 


The next two matrix transformations can be viewed geometrically. They reinforce the 
dynamic view of a matrix as something that transforms vectors into other vectors. 

1 0 0 " 

i 


Example 3 If A = 


0 1 0 
0 0 0 


then the transformationx —> Ax projects points in R 


onto the x,x 2 - coordinate plane because 


X, 


"1 

0 

O' 

X, 


X, 

X, 

-> 

0 

1 

0 

x^ 

= 

X, 

x 3 


0 

0 

0 

x 3 


0 



•x 2 


Example 4 Let A 


1 3 
0 1 _ 

called a shear transformation. 


A projection transformation 

., the transformation T \R 2 —> R z defined by T (x) = Ax is 
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The image of the point u = 


0 

2 


is 


T(u) = 


1 

0 


and the image of 


"2l 

is 

~1 

3" 

"2l 

= 

”8" 

2 


0 

1 

2 


2 



Here, T deforms the square as if the top of the square was pushed to the right while the 
base is held fixed. Shear transformations appear in physics, geology and crystallography. 


X 2 X 2 



A shear transformation 


Linear Transformations 


We know that if A is in x n matrix, then the transfonnation x —» Ax has the properties 
A(u + v) = Au + Av and A(cu) = cAu for all u, v in R" and all scalars c. 

These properties for a transformation identify the most important class of transfonnations 
in linear algebra. 

Definition A transfonnation (or mapping) T is linear if: 

1. T(u + v) = T(u) + T(v) for all u, v in the domain of T, 

2. T(cu) = cT(u) for all u and all scalars c. 

Example 5 Every matrix transfonnation is a linear transformation. 

Example 6 Let L :R J —» R 2 be defined by L(x,y,z ) = (x,y) . 

we let u = (x l ,y l ,z 1 ) and v = (x 2 ,y 2 ,z 2 ) . 

L(u + v) = L((x 1 ,y 1 ,z 1 ) + (x 2 ,y 2 ,z 2 )) 

= L(x 1 +x 2 ,y 1 +y 2 ,z 1 + z 2 ) 


©Virtual University Of Pakistan 


99 





9-Linear Transformations 


VU 


= (x l +x 2 ,y l +y 2 ) 

= (*i, T,) + (* 2 , v 2 ) = L(u) + Z(v) 

Also, if k is a real number, then 

Z(k«) = L(kx l ,ky l ,kz l ) = ( kx l ,ky l ) = kZ(«) 

Hence, L is a linear transformation, which is called a projection. The image of the vector 
(or point) (3, 5, 7) is the vector (or point) (3, 5) in xy-plane. See figure below: 



Geometrically the image under L of a vector (a, b, c) in R 3 is (a, b) in R 2 can be found by 
drawing a line through the end point P(a, b, c) of u and perpendicular to R 2 , the xy-plane. 
The intersection Q(a, b)of this line with the xy-plane will give the image under L. See the 
figure below: 
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Example 7 Let Z : i? —» i? be defined by L(x) = x 2 
Let x and y in R and 

L{x + v) = (x + y ) 2 = x 2 + y 2 + 2 xy ^ x 2 + y 2 = L{x) + L(y) 
=> L(x + y) * L(x) + L(y) 

So we conclude that the function L is not a linear transformation. 


Linear transformations preserve the operations of vector addition and scalar 
multiplication. 


Properties 

If T is a linear transformation, then 

1. T(0) = 0 

2. T(cu +dv) = cT(u) + clT(v) 

3. T(dV 7 + ...+ CpVp) =c 1 T(v 1 ) + ... + c p T(Vp) 

for all vectors u, v in the domain of T and all scalars c, d. 


Proof 

1. By the definition of Linear Transformation we have T(cu) = cT(u) for all u and all 
scalars c. Put c- 0 we’ll get T(0u)= 0T(u) This implies T(0) = 0 

2. Just apply the definition of linear transformation, i. e 

T(cu + dv) = T(cu) + T(dv) = cT(u) + dT(v) 


Property (3) follows from (ii), because T(0) = T (Ou) =0T(u) = 0. 
Property (4) requires both (i) and (ii): 


OBSERVATION Observe that if a transformation satisfies property 2 for all u, v and c, 
d, it must be linear (Take c = d = 1 for preservation of addition, and take d = 0) 
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3. Generalizing Property 2 we’ll get 3 


T(avi + ...+ c p v p ) = cjT(v0 + ... + c p T(v p ) 


Applications in Engineering 

In engineering and physics, property 3 is referred to as a superposition principle. 

Think of V/, ..., v p as signals that go into a system or process and T(vi), ..., T(v p ) as the 
responses of that system to the signals. The system satisfies the superposition principle if 
an input is expressed as a linear combination of such signals, the system’s response is the 
same linear combination of the responses to the individual signals. 


Example 8 Given a scalar r, define T : R —» R by 

T (x) = x+1. 

T is not a linear transfonnation (why!) because T(0) ^ 0 (by property 3) 

Example 9 Given a scalar r, define T : R 2 —> R 2 by T (x) = rx. 

T is called a contraction when 0 < r < 1 
and a dilation when r > 1. 


Let r = 3 and show that T is a linear transformation. 
Solution Let u, v be in R 2 and let c, d be scalars, then 


T (cu + dv) = 3 (cu + dv) 

= 3cu + 3dv 
= c (3u) + d(3v) 
= cT (u) + dT (v) 


Definition of T 
Vector arithmetic 


Thus T is a linear transformation because it satisfies (4). 


Example 10 Define a linear transformation T \R 2 —> R 1 by 


T(x) = 


o 

1 

i_ 


i 

1 

X 

to 

_1 

1 0 

i- 

i_ 


1 

J-T 

_i 


Find the images under T of u = 


4 

, v = 

2 

, and u + v = 

6 

1 


3 


4 
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"0 

-f 

~4~ 


"-1 

1 

"0 -f 

"2l 


"-3" 

Solution T(u) = 




= 



T(v) = 



= 



1 

0 

1 


4 

J 

1 0 

3 


2 



"0 

-f 

"6" 

r 

-4~ 




T(u + v) = \ 




= 







1 

0 

4 


6 





In the above example, T rotates u, v and u + v counterclockwise through 90°. 

In fact, T transforms the entire parallelogram detennined by u and v into the one 
determined by T (u) and T (v) 

Example 11 Let L: R —» R be a linear transformation for which: 

L (1, 0, 0) = (2, -1), 

L (0, 1, 0) = (3, 1), and 
L (0, 0, 1) = (-1, 2). 

Then find L (-3, 4, 2). 

Solution Since (-3, 4, 2) = -3i + 4j + 2k, 

Z(-3,4,2) = L(-3i + 4j + 2k) = -3 L(i) + 4 L(j) + 2 L(k) 

= — 3(2,—1) + 4(3,1) + 2(—1,2) = (4,11) 


Exercise 


1. Suppose that T : R —» R 2 and T(x) = Ax for some matrix A and each x in R\ How 
many rows and columns do A have? 


2 . 


Let A = 


1 

0 


. Give a geometric description of the transformation x —> Ax . 


3. The line segment from 0 to a vector u is the set of points of the form tu, where 
0 < t < 1. Show that a linear transformation T maps this segment into the segment 
between 0 and T(u). 


4. Let A = 


and T (v). 


1 

o 

o 

<N 
l_ 


' 1 ' 


“5 ' 

0 2 0 

,u = 

0 

,and 

-1 

0 0 2 


-3 


4 


. Define T:R 3 —>R 3 by T (x) = Ax. Find T (u) 


In exercises 5 and 6, with T defined by T (x) = Ax, find an x whose image under T is b, 
and detennine if x is unique. 
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5. A = 


' 1 

0 -ll 


"0 “ 

3 

1 -5 

,b = 

-5 

-4 

2 1 


-6 


1 

0 

3 


1 

0 

1 

-4 

,b = 

-5 

3 

2 

1 

-7 

-2 

-1 

-2 


3 


Find all x in R 4 that are mapped into the zero vector by the transfonnation x—» Ax. 


“l 

2 

-7 

5“ 











"l 

3 

4 

-3“ 

0 

1 

-4 

0 

8 . 

0 

1 

3 

-2 

1 

0 

1 

6 


3 

7 

6 

-5 

2 

-1 

6 

8 _ 







1 


9. Let b = 


-1 


and let A be the matrix in exercise 8. Is b in the range of the linear 


V J 

transfonnation x —> Ax? 


10. Let b = 


9 

5 

0 

-9 


and let A be the matrix in exercise 7. Is b in the range of the linear 


transfonnation x —> Ax? 


Let T (x) = Ax for x in R '. 

(a) On a rectangular coordinate system, plot the vectors u, v, T (u) and T (v). 

2 

(b) Give a geometric description of what T does to a vector x in R". 



~-l O' 


"5l 


"3 " 

11 . A = \ 

0 -1 

,u = 

2 

, and v = 

-1 



'.5 

O' 


"4l 


"-5" 

12. A = 

0 

.5j 

,u = 

2 

, and v = 

-2 


2 2 * 

13. Let T: R" —>R“ be a linear transformation that maps 


u = 


into 


and maps v = 


into 


1 

-4 


. Use the fact that T is linear find the images 


under T of 2u, 3v, and 2u + 3v. 
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14. Let e, = 


5 u 


'Ll = 


3 

-5 


, and y 2 = 


-2 

7 


. Let T: R 2 —> R 2 be a linear 


transfonnation that maps ei into yi and maps C 2 into yj. Find the images of 


and 


15. Let 
maps x 



X. 


"-7l 


"3 


X = 

1 

,Vj = 


and v 2 = 


. LetT: R“^R 2 


_X 2 J 


4 


_- 8 _ 



into jCjVj + . Find a matrix A such that T (x) is 


be a linear transformation that 
Ax for each x. 
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Lecture 10 

The Matrix of a Linear Transformation 


Outline 


• Matrix of a Linear Transformation. 

• Examples, Geometry of Transformation, Reflection and Rotation 

• Existence and Uniqueness of solution of T(x)=0 


In the last lecture we discussed that every linear transfonnation from R to R is actually 
a matrix transformation x —» Ax , where A is a matrix of order m x n . First see an example 



"1 O' 


'll 


"0" 

J_ The columns of I 2 = 

are e, = 


andcy = 


0 1 

1 

0 

Z 

1 


Suppose T is a linear transformation from R 2 into R 3 such that 


T(e 1 ) = 

' 5 ' 

-7 

and T(e 2 ) = 

"-3" 

8 


2 


0 


with no additional infonnation, find a formula for the image of an arbitrary x in R 2 
Solution Letx = 


X\ 


'll 


" 0 " 


= x, 


+ x 0 


x 2 

t 

0 _ 

z 

1 


= x x e x + x 2 e 2 


Since T is a linear transformation, T(x) = x l T(e l ) + x 2 T(e 2 ) 


T (x) = Xj 


' 5 ' 


"-3l 


5xj - 3 x 2 

-7 

+ x 2 

8 

= 

-7xj +8x 2 

2 


0 


2xj +0 


Hence 


T(x) = 


5Xj - 3 x 2 
-7xj + 8x 2 
2 x, + 0 


Theorem Let T : R" —> R'“ be a linear transformation. Then there exists a unique matrix 
A such that T (x) = Ax for all x in R" 
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In fact, A is the m x n matrix whose jth column is the vector T (ej), where e j is the jth 
column of the identity matrix in if". 

A = [ T (eJ ... T(e„)} 


Proof Write 



*1 


*1 


“0” 



“o“ 


V 


~0~ 



"0" 

X = 

x 2 

= 

0 

+ 

x 2 

+ ., 

..+ 

0 

= *1 

0 

+ x 2 

1 

+ ., 

■• + x„ 

0 


_ x »_ 


0_ 


0_ 





0_ 


0_ 



1 


= x l e 1 +... + x n e n =[e 1 ... e n 


Since T is Linear, So 


= h - e nV 


T(x) = T(x l e l +... + x n e n ) = x l T( < e l ) + ... + x ri T (e n ) 



= Ax 


( 1 ) 


The matrix ,4 in (1) is called the standard matrix for the linear transformation T. We 

know that every linear transformation from if" to if"' is a matrix transformation and vice 
versa. 

The term linear transformation focuses on a property of a mapping, while matrix 
transfonnation describes how such a mapping is implemented, as the next three examples 
illustrate. 

Example 2 Find the standard matrix A for the dilation transformation T (x) = 3x, x e R 1 . 


Solution Write 


T(e l ) = 3e l 


3 

0 


and T(e 2 ) = 3e 2 



0 

3 


0 

3 
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f 

Example 3 Let L:R~ —» R } is the linear operator defined by L 

v 

Find the standard matrix representing L and verify L (x) = Ax. 

Solution 

The standard matrix A representing L is the 3x3 matrix whose columns are L (e i), L 
(e 2 ), and L (e$) respectively. Thus 


X 



x + y 

y 


= 

y~ z 

z 

) 


x + z 



r 

V 

X 


"1 + 0" 


"1" 

L(e l ) = L 


0 


= 

o 

1 

o 

= 

0 


V 

0 

) 


i+oJ 


1 


f 

" 0 " 



"0 + 1" 


"1" 

L{e 2 ) = L 


1 


= 

1-0 

= 

1 


V 

0 

y 


0 + 0 


0 


f 

"o' 

X 


"0 + 0" 


"0 

L(e 3 ) = L 


0 


= 

0-1 

= 

-1 


k 

1 

J 


0 + 1 


1 


= col x {A ) 


= col 2 (A) 


= col 2 (A ) 




"1 

1 

0 " 



Hence 

A = 

0 

1 

-1 





1 

0 

1 





"1 

1 

0 " 

X 


x + y 

Now 

Ax = 

0 

1 

-1 

y 

= 

y~ z 



1 

0 

1 

z 


x + z 


= L(x) 


Hence verified. 

Example 4 Let T :R 2 —> R 2 be the transformation that rotates each point in R 2 
through an angles, with counterclockwise rotation for a positive angle. We could show 
geometrically that such a transfonnation is linear. Find the standard matrix A of this 
transfonnation. 



'f 

rotates into 

cos (p 

, and 

"O' 

rotates into 

-sin cp 

Solution 

o_ 

1 



sin<p 



cos cp 


See figure below. 
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By above theorem 


A = 


COS (p 

sin cp 


-sin cp 
cos cp 



A rotation transformation 


Example 5 A reflection with respect to the jc-axis of a vector u in R' is defined by the 
linear operator L(u) = L 


r 

r - 

> 


- 

a x 


a x 


i 

& 

to 

1 _ 

J 


i 

I 

& 

to 

1 _ 



/ 

T 

\ 

V 

r 

"0" 

\ 

"0 " 

Then 

L(e,) = L 

\ 

0 

) 

o_ 

and L(e 0 ) = L 

\ 

1 

) 

-1 


Hence the standard matrix representing L is A = 


1 0 
0 -1 


Thus we have L(u) = Au = 


1 

0 " 

a \ 


a \ 

0 

-1 

_a 2 \ 


_ a 2 _ 


To illustrate a reflection with respect to the x-axis in computer graphics, let the triangle T 
have vertices (-1, 4), (3, 1), and (2, 6). 


To reflect T with respect to x-axis, we let u x = 



, u~. 


2 

6 


and compute the 


images L (u /), L (u 2 ), and L (u 3 ) by forming the products 
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"1 0 " 

"-ll 


~-l~ 

0 -1 

4 


-4 


Au 2 


Au 3 


"1 0 " 

"3l 


"3 " 

0 -1 

1 


-1 


1 

o 

1_ 

-1 

K> 

_ i 


1 

<N 
i _ 

1 

o 

1 _ 

i 

VO 
_1 


l 

VO 

1 

_ l 


Thus the image of Thas vertices (-1, -4), (3, -1), and (2, -6). 

Geometric Linear Transformations of R 2 


Examples 3-5 illustrate linear transformations that are described geometrically. In 
example 4 transformations is a rotation in the plane. It rotates each point in the plane 
through an angles. Example 5 is reflection in the plane. 


Existence and Uniqueness of the solution of T(x)=b 

The concept of a linear transformation provides a new way to understand existence and 
uniqueness questions asked earlier. The following two definitions give the appropriate 
terminology for transformations. 


Definition A mapping T : R n —» R m is said to be onto if'" if each b in if"' is the image of 
at least one x in if". 

OR 

Equivalently, T is onto if'" if for each b in if"' there exists at least one solution of 
T (x) = b. “Does T map if" onto if'"?” is an existence question. 

The mapping T is not onto when there is some b in if"' such that the equation T (x) = b 
has no solution. 

Definition A mapping T :R" —> R m is said to be one-to-one (or 1:1) if each b in if'" is 
the image of at most one x in if". 
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OR 


Equivalently, T is one-to-one if for each b in if'" the equation T (x) = b has either a 
unique solution or none at all, “Is T one-to-one?” is a uniqueness question. 

The mapping T is not one-to-one when some b in if"' is the image of more than one 
vector in if". If there is no such b, then T is one-to-one. 

Example 6 Let T be the linear transformation whose standard matrix is 


A = 


1 

0 

0 


-4 8 1 

2-13 
0 0 5 


Does T map R 4 onto if 5 ? Is T a one-to-one mapping? 

Solution Since A happens to be in echelon fonn, we can see at once that A has a pivot 
position in each row. 

We kn ow that for each b in if 5, the equation Ax = b is consistent. In other words, the 
linear transformation T maps R 4 (its domain) onto R 3 . 

However, since the equation Ax = b has a free variable (because there are four variables 
and only three basic variables), each b is the image of more than one x. That is, T is not 
one-to-one. 


Theorem Let T :R" —> R m be a linear transformation. Then T is one-to-one if and only if 
the equation T (x) = 0 has only the trivial solution. 

Proof: Since T is linear, T (0) = 0 if T is one-to-one, then the equation T (x) = 0 has at 
most one solution and hence only the trivial solution. If T is not one-to-one, then there is 
a b that is the image of at least two different vectors in if" (say, u and v). 

That is, T (u) = b and T (v) = b. 

But then, since T is linear T(u - v) = T(u) = b - b = 0 

The vector u - v is not zero, since u^v. Hence the equation T (x) = 0 has more than one 
solution. So either the two conditions in the theorem are both true or they are both false. 

Kernel of a Linear Transformation 

Let T : V —> W be a linear transformation. Then kernel of T (usually written as Ker7), 
is the set of those elements in V which maps onto the zero vector in W .Mathematically: 
KerT = {v<=V\ T(v) = 0 in Wj 
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Remarks 

i) Ker7 is subspace of V 

ii) T is one-one iff KerT = 0 in V 


One-One Linear Transformation 

Let T:V —>W be a linear transformation. Then T is said to be one-one if for any 
u, v e V with u ^ v implies Tu^Tv. Equivalently if Tu = Tv then u = v. 

T is said to be one-to-one or bijective if 

i) T is one-one 

ii) T is onto 

Theorem Let T : R" —» R m be a linear transformation and let A be the standard matrix for 
T Then 


(a) T maps if" onto if'" if and only if the columns of A span if"'; 

(b) T is one-to-one if and only if the columns of A are linearly independent. 

Proof: 


(a) The columns of A span if'" if and only if for each b the equation Ax = b is consistent - 
in other words, if and only if for every b, the equation T(x) = b has at least one 
solution. This is true if and only if T maps if" onto if'". 

(b) The equations T (x) = 0 and Ax = 0 are the same except for notation. So T is one-to- 
one if and only if Ax = 0 has only the trivial solution. This happens if and only if the 
columns of A are linearly independent. 


We can also write column vectors in rows, using parentheses and commas. Also, 


we apply a linear transformation T to a vector - say, x = 


= (x 15 x 2 ) we 


when 

write 


T (jCj, x 2 ) instead of the more formal T ((x,, x 2 )). 


Example 7 Let T (x ly x 2 ) = (3x] +x 2 , 5x 2 + 7x 2 , x 2 + 3x 2 ). 

Show that T is a one-to-one linear transformation. 

Does T map R 2 onto if 5 ? 
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Solution When x and T (x) are written as column vectors, it is easy to see that T is 
described by the equation 


"3 f 



3xj + x. 


x i 



5 7 

x 2 

~ 

5Xj + 7x 2 

1 3 



Xj + 3x, 


so T is indeed a linear transformation, with its standard matrix A shown in (4). The 
columns of/I are linearly independent because they are not multiples. Hence T is one-to- 
one. To decide if T is onto R 3 , we examine the span of the columns of A. Since A is 3x2, 
the columns of A span R 3 if and only if A has 3 pivot positions. This is impossible, since 
A has only 2 columns. So the columns of A do not span R 3 and the associated linear 
transfonnation is not onto R 3 . 
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Exercises 


1. Let T : R 2 —» R be transformation that first performs a horizontal shear that maps 
e 2 into e 2 -. 5 e l (but leaves e x unchanged) and then reflects the result in the X2 - axis. 
Assuming that T is linear, find its standard matrix. 

Assume that T is a linear transfonnation. Find the standard matrix of T. 

2. T:R 2 ~^R\ 7(1,0) = (4,-1,2) and 7(0,1) = (-5,3,-6) 

3. T :R 3 — » R 2 ,T(e l ) = (1,4 ),T(e 2 ) = (-2,9 ),andT(e i ) = (3,-8), where ei, Q2, and e 3 are 
the columns of the identity matrix. 

4. T \ R 2 — » R 2 rotates points clockwise through n radians. 

5. T : R 2 —> R 2 is a “vertical shear” transformation that maps ei into ei + 2e2 but leaves 
the vector e 2 unchanged. 

6. T : R 2 — > R 2 is a “horizontal shear” transformation that maps e 2 into e 2 - 3ei but 
leaves the vector ei unchanged. 

7 . T : R 3 —> R 3 projects each point (xi, X2, X3) vertically onto the xiX2-plane (where 
X3=0). 


8. T : R 2 —» R 2 first perfonns a vertical shear mapping ei into ei - 3e2(leaving e 2 
unchanged) and then reflects the result in the x 2 -axis. 

9. T :R 2 —> R 1 first rotates points counterclockwise through n !A radians and then 
reflects the result in the x 2 -axis. 

Show that T is a linear transformation by finding a matrix that implements the mapping. 
Note that xi, X 2 , ... are not vectors but are entries in vectors. 

10. T(x 1 ,x 2 ,x 3 ,x 4 ) = (Xj +x 2 ,x 2 +x 3 ,x 2 +x 4 ,0) 

11. T(x l ,x 2 ,x 3 ) = (3x, -x 3 ,Xj +4x 2 +x 3 ) 

12. r(xj,x 2 ,x 3 ,x 4 ) = 3xj -4x 2 +8x 4 

13. Let T : R 2 —> R 2 be a linear transformation such that T (x,,x 2 ) = (x, +x 2 ,4xj +7x 2 ) . 
Find x such that T (x) =(-2, -5). 
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13. Let T : R 2 —>■ R J be a linear transformation such that 

T(x j,jc 2 ) = (jCj +2x 2 ,-Xj -3x 2 ,-3xj - 2 x 2 ) . Find jc such that T (x) =(-4, 7, 0). 

In exercises 14 and 15, let T be the linear transfonnation whose standard matrix is given. 

14. Decide if T is one-to-one mapping. Justify your answer. 

"-5 10-5 4" 

8 3-47 

4-95-3 
-3 -2 5 4 _ 

15. Decide if T maps R 5 onto R 5 . Justify your answer. 

“ 4 -7 3 7 5 “ 

6 -8 5 12 -8 

-7 10 -8 -9 14 

3 -5 4 2 -6 

-5 6-6-7 3 
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Lecture 11 


Matrix Operations 

(i-j)th Element of a matrix 

Let A be an m x n matrix, where m and n are number of rows and number of columns 
respectively, then a tj represents the i-th row and y-th column entry of the matrix. For 

example a l2 represents 1 st row and 2 nd column entry. 

Similarly a i2 represents 3 ld row and 2 nd column entry. The columns of A are vectors in 
R m and are denoted by (boldface) a 1 ,a 2 ,-,a n . 


These columns are A = [a, a 2 ... a n ] 

The number <x is the i-th entry (from the top) of j-th column vector o / . 

Column 


J 


Row i 


a 


ii • • • i j 


a,. ... a 


1 n 


a, ... a . a 

1 1 ij in 


a m\ •" a mj •" a mn 

t t t 


= A 


a, 


a. 


a.. 


Figure 1 Matrix notation. 


Definitions 

A diagonal matrix is a square matrix whose non-diagonal entries are zero. 

k, o ... oi 


The diagonal entries in A = [«, y ] are a u ,a 22 ,a 33 ,--- and they form the main diagonal 
of A. 
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For example 







'1 

0 

0 

0 ~ 



'2 

0 

0 






“5 

0 


0 

3 

0 


0 

0 

0 

0 

0 

7 






0 

0 

16 

0 




0 

0 

11 










— 


0 

0 

0 

0 


are all diagonal 


matrices. 

Null Matrix or Zero Matrix 

An m x n matrix whose entries are all zero is a Null or zero matrix and is always written 
as O. A null matrix may be of any order. 


For example 


3x3 

are all Zero Matrices 









'0 

0 

0 

0 

0 " 

"0 

0 

o' 


"0 

o' 


0 

0 

0 

0 

0 

0 

0 

0 


0 

0 


0 

0 

0 

0 

0 

0 

0 

0 


0 

0 











“ 



0 

0 

0 

0 

0 


3x2 


4x5 


Equal Matrices 

Two matrices are said to be equal if they have the same size (i.e., the same number of 
rows and columns) and same corresponding entries. 

Example 1 Consider the matrices 



'2 1 


'2 1" 


'2 

1 0" 

A = 

3 x + l_ 

B = 

3 5 

C = 

_3 

4 °_ 


The matrices A and B are equal if and only if x+1 = 5 or x = 4. There is no value of x for 
which A = C, since A and C have different sizes. 


If A and B are m x n matrices, then the sum, A + B, is the m x n matrix whose columns 
are the sums of the corresponding columns in A and B. Each entry in A + B is the sum of 
the corresponding entries in A and B. The sum A + B is defined only when A and B are of 
the same size. 


If r is a scalar and A is a matrix, then the scalar multiple rA is the matrix whose columns 
are r times the corresponding columns in A. 


Note: Negative of a matrix A is defined as - A to mean (—1 )A and the difference of A and 
B is written as A—B, which means A + (-1) B. 


Example 2 


Then 


Let A = 


A + B = 


4 0 5 
-13 2 

5 1 6~ 
2 8 9 



'1 1 1] 


ro 

1 

<N 

B = 

3 5 7 

, c = 

0 1 
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But A + C is not defined because A and C have different sizes. 


2B = 2 


A-2B 


"1 1 ll 


'2 

2 

2~ 

3 5 7 


6 

10 

14 


O 

1_ 

5“ 


'2 

2 

2~ 

-1 3 

2 


6 

10 

14 


2 -2 
-7 -7 


3 

-12 


Theorem 1 Let A, B, and C are matrices of the same size, and let r and s are scalars. 


a. 

A+B=B+A 

d. 

r (A + B)= r A + r B 

b. 

(A +B) + C = A+(B + C) 

e. 

(r + s) A = r A + s A 

c. 

A + 0 =A 

f. 

r (sA) = (rs) A 


Each equality in Theorem 1 can be verified by showing that the matrix on the left side 
has the same size as the matrix on the right and that corresponding columns are equal. 
Size is no problem because A, B, and C are equal in size. The equality of columns 
follows immediately from analogous properties of vectors. 

For instance, if the jth columns of A, B, and C are a ., h j and c ., respectively, then the 
jth columns of (A + B) + C and A + (B + C) are 

(a. + bj) + Cj and a . + {b j + c .) 
respectively. Since these two vector sums are equal for each j, property (b) is verified. 

Because of the associative property of addition, we can simply write A + B + C for the 
sum, which can be computed either as (A + B) + C or A + (B + C). The same applies to 
sums of four or more matrices. 

Matrix Multiplication 

Multiplying an mxn matrix with an nxp matrix results in an mxp matrix. If many 
matrices are multiplied together, and their dimensions are written in a list in order, e.g. 
mxn, nxp, pxq, qxr, the size of the result is given by the first and the last numbers (mxr). 

It is important to keep in mind that this definition requires the number of columns of the 
first factor A to be the same as the number of rows of the second factor B. When this 
condition is satisfied, the sizes of A and B are said to conform for the product AB. If the 
sizes of A and B do not conform for the product AB, then this product is undefined. 

Definition If A is an mxn matrix, and if B is an nxp matrix with columns b x ,---,b p , 
then the product AB is the m x p matrix whose columns are Ab x , • • •, Ab p . 

That is AB = A^bj b 2 ... Z^] = [v4hj Ab 2 ... 
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This definition makes equation (1) true for all x in R p . Equation (1) proves that the 
composite mapping (AB) is a linear transformation and that its standard matrix is AB. 
Multiplication of matrices corresponds to composition of linear transfonnations. 


A convenient way to determine whether A and B conform for the product AB and, if so, 
to find the size of the product is to write the sizes of the factors side by side as in Figure 
below (the size of the first factor on the left and the size of the second factor on the 
right). 

A B = AB 

m x s s x n m x n 

t t Inside f t 


Outside 


If the inside numbers are the same, then the product AB is defined and the outside 
numbers then give the size of the product. 


Example 3 Compute AB , where A = 


2 

1 




3 

-2 


6 

3 


Solution: Her eB = [bj b 2 b 3 ], therefore 



"2 3 " 

"4l 


"2 3 ' 

" 3 " 


"2 3 " 

"6" 

Ah = 



, Ah = 



, Ah = 



1 

1 -5 

1 

7 Z 

1 -5 

-2 

1 -5 

3 


Then 


'll] 


"O' 


'21" 

-1 


!3_ 


-9 


AB = A[b j b 2 h 3 ] = 


11 0 21 
13 -9 

t t 

Abi Ab2 Ab 3 


Note from the definition of AB that its first column, Abi, is a linear combination of the 
columns of A, using the entries in b 3 as weights. The same holds true for each column of 
AB. Each column of AB is a linear combination of the columns of A using weights from 
the corresponding column of B. 
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Example 4 Find the product AB for 







'4 

1 

4 

3" 


'1 2 

4] 







A = 

2 6 

0 

and 

B = 

0 

-1 

3 

1 



— 



2 

7 

5 

2 


Solution It follows from definition that the product AB is formed in a column-by-column 
manner by multiplying the successive columns of B by A. The computations are 


Similarly, 





~4~ 

"1 

2 

4 


0 

2 

6 

0 _ 







2 


2 

6 

2 

6 


= 4Cj +0c 2 +2c 3 = (4) 


+ ( 0 ) 


+ ( 2 ) 


12 

8 


= ( 1 ) 


+ (-l) 


+ (7) 


27 

-4 


= (4) 


+ (3) 


+ (5) 


30 

26 


Thus, 


1 2 4 

2 6 0 


3 

1 

2 


= (3) 




13 

12 


4 14 3 



'1 2 

4 




“12 

27 

30 

13" 

AB = 

2 6 

0 

0 

'A 

-1 3 1 

n c 

— 

_ 8 

-4 

26 

12 


2 7 5 2 


Example 5 (An Undefined Product) Find the product BA for the matrices 







~4 

1 4 

3" 


“l 2 

4“ 






A = 

2 6 

0 _ 

and 

B = 

0 

-1 3 

1 






2 

7 5 

2 


Solution The number of columns of B is not equal to number of rows of A so BA 
multiplication is not possible. 

The matrix B has size 3x4 and the matrix A has size 2x3. The “inside” numbers are 
not the same, so the product BA is undefined. 

Obviously, the number of columns of A must match the number of rows in B in order for 
a linear combination such as Ab / to be defined. Also, the definition of AB shows that AB 
has the same number of rows as A and the same number of columns as B. 
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Example 6 If A is a 3 x 5 matrix and B is a 5x2 matrix, what are the sizes of AB and 
BA, if they are defined? 


Solution The product of matrices A and B of orders 3 x 5 an( j 5x2 w qi result in 

3 x 2 matrix AB. 

But for BA we have 2 an( j 3x5 ? here number of columns in 1st matrix are 2 which is 
not equal to number of rows in 2nd matrix. So BA is not possible. 

Since A has 5 columns and B has 5 rows, the product AB is defined and is a 3x2 

matrix: 

A B AB 







"* 

*- 



* 

* 

* 

* 



= 1 = 

* 


"* 


* 

* 

* 

* 

* 


* 

* 

= 

* 

* 

* 

* 

* 

* 

* 


* 

* 


* 

* 






* 

* 




Match 


Size of AB 


3x2 


The product BA is not defined because the 2 columns of B do not match the 3 rows of A. 

The definition of AB is important for theoretical work and applications, but the following 
rule provides a more efficient method for calculating the individual entries in AB when 
working small problems by hand. 

Row-Column Rule for Computing AB 

Explanation 

If a matrix B is multiplied with a vector x, it transforms x into a vector Bx. If this vector 
is then multiplied in turn by a matrix A, the resulting vector is A (Bx). 



x* • • 


Bx A(Bx) 

Multiplication by B and then A 

Thus A(Bx) is produced from x by a composition of mappings. Our goal is to represent 
this composite mapping as multiplication by a single matrix, denoted by AB, so that 
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A (Bx) = (AB) x 


( 1 ) 



by AB 

Multiplication by AB 


If A is m x n , B is n x p , and x is in R p , denote the columns of B by b 2 , • • •, b and the 
entries in x by Xj, • • •, x p , then Bx = x l b 1 + x 2 b 2 H-1- x p b p 

By the linearity of multiplication by A, 

A(Bx ) = A(x 1 b 1 ) + A(x 2 b 2 ) H-t- A(x p b p ) 

= x x Ab x + x 2 Ab 2 H-1- x p Ab p 

The vector A (Bx) is a linear combination of the vectors Ab x , • • •, Ab p , using the entries in 
x as weights. If we rewrite these vectors as the columns of a matrix, we have 

A(Bx) = \^Ab x Ab 2 ... Ab p Jx 

Thus multiplication by \^Ab } Ab , ... Ab p ~^ transforms x into A (Bx). 

We have found the matrix we sought! 

Row-Column Rule for Computing AB 

If the product AB is defined, then the entry in row i and column j of AB is the sum of the 
products of corresponding entries from row i of A and column j of B. If (AB) jj denotes 
the (i,j) - entry in AB. and if A is an in x n matrix, then 

( AB )ij = a a b x j + a i2 b 2J + ... + a in b nj 

To verify this rule, let B = ... b p , Column j of AB is Abj, and we can compute 

Abj. The ith entry in Abj is the sum of the products of corresponding entries from row i of 
A and the vector bj, which is precisely the computation described in the rule for 
computing the (i,j) - entry of AB. 
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Finding Specific Entries in a Matrix Product Sometimes we will be interested in 
finding a specific entry in a matrix product without going through the work of computing 
the entire column that contains the entry. 

Example 7 Use the row-column rule to compute two of the entries in AB for the 
matrices in Example 3. 

Solution: To find the entry in row 1 and column 3 of AB, consider row 1 of A and 

column 3 of B. Multiply corresponding entries and add the results, as shown below: 


I 


3 " 

"4 3 6" 


U 

□ 

2(6)+ 3(3) 


"□ □ 21 

-5 

1 -2 3 


.□ 

□ 

□ j 


.□ □ □. 


For the entry in row 2 and column 2 of AB, use row 2 of A and column 2 of B: 


i 



"2 3 " 

"4 3 6" 


"□ □ 21 


U □ 21 

-» 

! -5 

1 -2 3_ 


.□ 1(3) H—5(—2) □_ 


.□ 13 □. 


Example 8 Use the dot product rule to compute the individual entries in the product of 








~4 

1 4 

3" 



"l 2 

4] 






AB where 

A = 

2 6 

0_ 

and 

B = 

0 

-1 3 

1 







2 

7 5 

2 


Solution Since A has size 2x3 and B has size 3x4, the product AB is a 2x4 matrix of 
the form 

r ,(A) x c,(B) r/A) x c 2 (B) r,(A) x c 3 (B) r,(A) x c 4 (B)~ 

r 2 (A) x c,(B) r 2 (A) x c 2 (B) r 2 (A)*c 3 (B) r 2 (A)*c 4 (B) 

where i](A) and r 2 ( A) are the row vectors of A and c 1 (B),c 2 (B),c 2 (B) and c 4 (B) are 

the column vectors of B. For example, the entry in row 2 and column 3 of AB can be 
computed as 


1- 

"xt- 

<N 

l_ 

“4 14 3“ 

0-131 


'□ 

□ □ □' 

[2 6 0J 

2 7 5 2 


n 

□ Id □. 


(2 x 4) + (6 x 3) + (0 x 5) = 26 

and the entry in row 1 and column 4 of AB can be computed as 


©Virtual University Of Pakistan 


123 




11- Matrix Operations 


VU 


ri 2 4i h 1 h j [□ □ □ mr 

L 2 6 °J[ 2 7 5 2 J LD □ □ □. 

(Ix 3 ) + (2xl) + ( 4 x2) = 13 
Here is the complete set of computations: 

(AB) n =(1x4) + (2x0) + (4x2) = 12 

{AB) n = { Ixl) + (2x-l) + (4x7) = 27 
{AB) n =(1x4) + (2x3) + (4x5) = 30 
(AB) U = (1x3) +(2xl) +(4x2) = 13 
(AB) n = (2 * 4) + (6 x 0) + (0 x 2) = 8 
(AB) 22 =(2xl) + (6x-l) + (0x7) = -4 
(AB) 2 3 =(2x4) + (6x3) + (0x5) = 26 
(AB) 24 =( 2x3) + (6x1) + (0x2) = 12 


Finding Specific Rows and Columns of a Matrix Product 

The specific column of AB is given by the formula 

AB = A[b 1 b 2 ••• b n } = [Ab x Ab 2 - Ab n ] 



ci i 


cijB 

Similarly, the specific row of AB is given by the formula AB = 

a 2 

B = 

a 2 B 




a B 

m 


Example 9 Find the entries in the second row of AB, where 



' 2 

-5 

0 ' 









~4 

-6 


-1 

3 

-4 




A = 

6 

-8 

-7 

, B = 

7 

1 






3 

2 


-3 

0 

9 





Solution: By the row-column rule, the entries of the second row of ^42? come from row 2 
of A (and the columns of B ): 


1 

o 

1 

<N 

l_ 

“4 -6 


□ □ _ 
-4 + 21-12 6 + 3-8 


'□ □" 
5 1 

-1 3 -4 

6 -8 -7 

-3 0 9 

7 1 

3 2 


1 

□ □ 

□ □ 

1_ 


□ □ 

□ □ 

1_ 
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Example 10 (Finding a Specific Row and Column of AB) 








~4 

1 

4 

3~ 



“l 2 

4“ 







Let 

A = 

2 6 

0 

and 

B = 

0 

-1 

3 

1 







2 

7 

5 

2 


Find the second column and the first row of AB. 


Solution 


c 2 (AB) = Ac 2 (B) = 


1 2 4 

2 6 0 


1 

-1 

7 


27 

-4 


r 1 (AB) = r I (A)B = [ 1 2 


4 ] 


4 14 3 
0-131 
2 7 5 2 


[12 27 30 13] 


Properties of Matrix Multiplication 

These are standard properties of matrix multiplication. Remember that 7 m represents the 
m x m identity matrix and I m x = x for all x belong to if"'. 


Theorem 2 Let A be in x n , and let B and C have sizes for which the indicated sums and 


products are defined. 

a. A (BC) = (AB) C 

(associative law of multiplication) 

b. 

A (B + C) =AB +AC 

(left distributive law) 

c. 

(B + C)A = BA + CA 

(right distributive law) 

d. 

r (AB) = (r A)B = A(r B) 

(for any scalar r) 

e. 

I m A = A = Al m 

(identity for matrix multiplication) 


Proof. Properties (b) to (e) are considered exercises for you. We start property (a) 
follows from the fact that matrix multiplication corresponds to composition of linear 
transfonnations (which are functions), and it is known (or easy to check) that the 
composition of functions is associative. 


Here is another proof of (a) that rests on the “column definition” of the product of two 


matrices 


. Let C = 


c, 


c, 


By definition of matrix multiplication BC — ^Bc l ... Bc ~] 

A(BC) = [A(Bc x ) ... A(Bc p )] 

From above, we know that A(Bx) = (AB)x for all x, so 

A{BC) = \(AB)c x ... (H5)c p ] = (H5)C 


The associative and distributive laws say essentially that pairs of parentheses in matrix 
expressions can be inserted and deleted in the same way as in the algebra of real 
numbers. In particular, we can write ABC for the product, which can be computed as 
A(BC) or as (AB)C. Similarly, a product ABCD of four matrices can be computed as 
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A(BCD) or (ABC)D or A(BC)D, and so on. It does not matter how we group the matrices 
when computing the product, so long as the left-to-right order of the matrices is 
preserved. 

The left-to-right order in products is critical because, in general, AB and BA are not the 
same. This is not surprising, because the columns of AB are linear combinations of the 
columns of A, whereas the columns of BA are constructed from the columns of B. 


Example 11 Let A = 


If AB = BA, we say that .4 and B commute with one another. 

5 1 1 [2 O' 

and B = 

_3 -2J |_4 3 

Show that these matrices don not commute, i.e. AB ^ BA. 

Solution: 

AB = 


BA = 


For emphasis, we include the remark about commutativity with the following list of 
important differences between matrix algebra and ordinary algebra of real numbers. 


“5 1 


"2 0“ 


“10 + 4 0 + 3“ 


14 3 “ 

_3 -2 


_4 3_ 


6-8 0-6 


1 

SO 

1 

(N 

1 

_l 

"2 0“ 


5 1 “ 


“10 + 0 2-0“ 


"10 2 “ 

4 3_ 


3 -2_ 


20+9 4-6 


29 -2 


WARNINGS 

1. In general, AB + BA . Clear from the Example #11. 

2. The cancellation laws do not hold for matrix multiplication. That is, if 
AB = AC , then it is not true in general that B = C. 

For example: Consider the following three matrices 



“-3 

2“ 


“-1 2 “ 


r-i 2 1 

A = 

-6 

4 

B = 

3 -2 

C = 

i 

<N 

1 

m 

_i 


AB = 


9 

18 



But B + C 


3, If a product AB is the zero matrix, you cannot conclude in general that 
either A = 0 or B = 0. 


For example: 


IfAB = 


0 

0 


0 

0 


then it can be either 
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Example 


AB = 


AB = 



'1 

4] 



"0 

0" 

A = 

6 

2 

and 

B = 

0 

0 


"0 

O' 



'1 

4~ 

A = 

0 

0 

and 

B = 

6 

2 


5 

1 ' 

"2 

O ' 


"14 3 " 

3 

-2 

4 

3 


1 

VO 

1 

(N 

1 

_ I 


5 1 " 

' 3 O' 


"15-1 0 + 3" 


"14 3 ' 

3 -2 

-1 3_ 


6-8 0-6 


1 

VO 

1 

<N 

1 

_l 


Powers of a Matrix If A is an n x n matrix and if A: is a positive integer, A 1 ' denotes the 
product of k copies of.4, A k = A...A Also, we interpret^ 0 as I. 

k 

Transpose of a Matrix Given an m x n matrix A, the transpose of A is the n x m matrix, 
denoted by A*, whose columns are formed from the corresponding rows of A. 

OR, if A is an m x n matrix, then transpose of A is denoted by A*, is defined to be the nxm 
matrix that is obtained by making the rows of A into columns; that is, the first column of 
^ ? is the first row of A, the second column ofz^is the second row of A, and so forth. 


Example 12 (Transpose of a Matrix) 

The following is an example of a matrix and its transpose. 


A = 

A 1 = 


2 3 
1 4 
5 6 

~2 1 5 

3 4 6 





"-5 

2 ' 



a b 





Example 13 Let A = 

c d 

, B = 

1 

-3 

, c = 


~ 


0 

4 




A‘ = 

a c 


'-5 1 0" 

Then 

b d\ 

, B‘ = 

2-3 4 



1 1 1 

5-2 7 

-3" 

5 

-2 

7 


Theorem 3 Let A and B denote matrices whose sizes are appropriate for the following 
sums and products. 
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a. (A')' = A 

b. (A + B)' =A' +B' 

c. For any scalar r, (rA)' = rA‘ 

d. (AB)‘=B , A‘ 

The generalization of (d) to products of more than two factors can be stated in words as 
follows. 

“The transpose of a product of matrices equals the product of their transposes in 
the reverse order.” 
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Lecture 12 

The Inverse of a Matrix 


In this lecture and the next, we will consider only square matrices and we will investigate 
the matrix analogue of the reciprocal or multiplicative inverse of a nonzero real number. 

Inverse of a square Matrix 

If A is an n x n matrix, A matrix C of order n x n is called multiplicative inverse of A if 
AC = CA = I where / is the «x« identity matrix. 

Invertible Matrix 

If the inverse of a square matrix exists, it is called an invertible matrix. 

In this case, we say that ,4 is invertible and we call C an inverse of A 
Note : If B is another inverse of A, then we would have 
B = BI = B(AC) = (BA)C = IC = C. 


Thus when A is invertible, its inverse is unique. 

The inverse of A is denoted by A' 1 , so that 

AA~ l = I and A^A = I 

Note: A matrix that is not invertible is sometimes called a singular matrix, and an 
invertible matrix is called a non-singular matrix. 


Warning 

By No means A '.A = A.A 1 = I The identity matrix A 1 
real number, we have3~‘ = —. 


—, as in case of 
A 


A 1 is, in fact the n x n matrix corresponding to the n x n matrix A, which satisfies the 
property 

A .A = A. A 1 = / The identity matrix 
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Example 1 If A = 


AC = 


2 5 

-3 -7 


and C = 


-7 -5 
3 2 


, then 


CA = 


2 5 ~ 

1 

1 

r- 

1 

i_ 


'-14 + 15 -10 + 10 " 


"1 0 " 

l 

1 

l —] 

L 3 2 j 


21-21 15-14 


0 1 

1 

-4 

1 

"2 5 " 


'-14 + 15 -35 + 35 " 


"1 0 " 

3 2 

-3 -7 


6-6 15-14 


0 1 


and 


Thus C =A~ 1 . 


Theorem 


Let A = 


a b 
c d 


If ad -be * 0 , then A is invertible or non singular and A 1 = 
If ad -be = 0 , then A is not invertible or singular. 


1 


ad -be 


d -b 
-c a 


The quantity ad-be is called the determinant of A, and we write 
det^4 = ad-be 


This implies that a 2 x 2 matrix A is invertible if and only if det A * 0. 
Example 2 Find the inverse of A = 


3 4 
5 6 


Solution We have det ,4 = 3(6) - 4(5) = -2*0. 


1 


Hence A is invertible A = — 

-2 


" 6 - 4 " 


" 6 /(— 2 ) — 4 /(— 2)~1 


" -3 2 

-5 3 


5 /(— 2 ) 3 /(— 2 ) _ 


_ 5/2 —3 / 2 _ 


The next theorem provides three useful facts about invertible matrices. 

Theorem 

a. If A is an invertible matrix, then A' 1 is invertible and (A 1 ) 1 = A 

b. If A and B are n x n invertible matrices, then so is AB, and the inverse of AB is 
the product of the inverses of A and B in the reverse order. That is 
(AB)- 1 = BT X A~ X 


c. 


If A is an invertible matrix, then so is A 7 , and the inverse of A 7 is the transpose of 
yT 7 . That is ( A T y 1 =(A~ 1 ) T 
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Proof 

(a) We must find a matrix C such that A l C = I and CA 1 = / 

However, we already know that these equations are satisfied with A in place of C. Hence 
A' 1 is invertible and A is its inverse. 

(b) We use the associative law for multiplication: 

(AB)(B 'A 1 ) = A(BB ')A 
= AI A' 

= AA~ X 
= 1 


A similar calculation shows that (B l A l )(AB) = I. 

Hence AB is invertible, and its inverse is B l A~ l i.e {AB )= B~ l A~ l 

Generalization 

Similarly we can prove the same results for more than two matrices i.e 


The product of nxn invertible matrices is invertible, and the inverse is the product of 
their inverses in the reverse order. 


Example 3 (Inverse of a Transpose). Consider a general 2x2 invertible matrix and its 
transpose: 


A = 


a b 
c d 


and 


A' = 


a c 
b d 


Since A is invertible, its detenninant {ad-be) is nonzero. But the determinant of A‘ is 
also (ad - be ), so A 1 is also invertible. It follows that 

d c 


(Ay 1 = 


ad -be 
b 


ad -be 
a 


Now 


A = 


ad -be ad - be 
d b 


-( 1 ) 


ad -be 
c 


ad -be 
a 


ad -be ad - be 
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Therefore, 


(A- 1 )' = 


d 


ad-be ad-be 

b a 


ad -be ad - be 


-( 2 ) 


From (1) and (2), we have 


(A‘)-'= (A-')'. 


Example 4 (The Inverse of a Product). Consider the matrices 

"3 2" 

2 2 


A = 

AB = 


Here 


1 2 
1 3 
7 6 
9 8 


B = 


(AB)- 1 = —-— Adj(AB) = — 
\AB 1-2 


A 1 = —Adj (A) = - 
\A\ 1 


3 -2 

-1 1 


B~ l =-Adj(B) = - 
\B\ 2 


" 8 

-6l 


1 

m 

1 

-9 

7 


9 7 

2 2 J 


" 3 

-2 



-1 

1 



_ 

' 


-2 

1 

'll 


B l A~' = 

Thus, (AB)-'=B l A~ l 


1 -1 

- 1 f 


-2 3 

3 -2 

-1 1 


- 1 f 


4 -3 

-1 l 
2 2 


Theorem: If A is invertible and n is a non-negative integer, then: 

(a) A" is invertible and (A 11 )- 1 =A~ n = (A^ 1 )" 

(b) kA is invertible for any nonzero scalar k, and (kAf 1 = k~'A~ ] . 


Example 5 (Related to the above theorem) 

(a) Let 

"1 2 


Now 

Also, 


(b) 


A = 


1 3 



i 

1 

" 3 

-2' 


" 3 -2] 


then A 

1 =- Adi(A) = - 



= 




Ml 

1 

-1 

1 


-1 1 


( \ \3 

" 3 -2" 

" 3 -2" 

" 3 

-2" 


f 41 -301 

(a~ 1 ) = 





= 



V / 

-1 1 

-1 1 

-1 

1 


L-i5 ii J 


A 3 = 


"1 2 

"1 2 

"1 2] 


"11 

30' 





1 3 

1 3 

1 3_ 


15 

41 





-1 

1 


f 

" 41 

-30' 


' 41 

-30" 

(11)(41) — 

(30)(15) 

-15 

11 


-15 

11 


1 \3 


= (A- 1 ) 
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Take A= 


kA=3A= 




1 2 
3 1 


and k=3 


“3 6" 

(lrAY X ~ 13 'I'T 1 - 1 

" 3 

-6l 


—1/15 

2/15 " 

9 3 

, ) — l J/i ) — 

9-54 

-9 

3 


_ 1/5 

— 1/15 


-( 1 ) 


1 -2 
-3 1 


A=So k A =3 A =-,— 


1 -1 

" 1 

-2] 


—1/15 

2/15 " 

3'T 

-3 

1 


1/5 

-1/15 


-( 2 ) 


From (1) and (2), we have 
(3 Ay 1 =3 "'A' 1 


There is an important connection between invertible matrices and row operations that 
leads to a method for computing inverses. As we shall see, an invertible matrix A is row 
equivalent to an identity matrix, and we can find A' 1 by watching the row reduction of A 

to I. 

Elementary Matrices 

As we have studied that there are three types of elementary row operations that can be 
performed on a matrix: 

There are three types of elementary operations 

• Interchanging of any two rows 

• Multiplication to a row by a nonzero constant 

• Adding a multiple of one row to another 

Elementary matrix 

An elementary matrix is a matrix that results from applying a single elementary row 
operation to an identity matrix. 


Some examples are given below: 




“1 

0 

0 

0“ 














1 

0 

3 


“1 

0 

0“ 

“l 

0 


0 

0 

0 

1 


0 

1 

0 


0 

1 

0 

0 

-3 


0 

0 

1 

0 










- 1 







0 

0 

1 


0 

0 

1 



0 

1 

0 

0 










Multiply the Interchange the Add 3 times the Multiply the 

second row second and third row of l 3 to first row of 

of l 2 by -3. fourth rows of l 4 . the first row. I 3 by 1. 

From Def it is clear that elementary matrices are always square. 

Elementary matrices are important because they can be used to execute elementary row 
operations by matrix multiplication. 
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Theorem If A is an n x n identity matrix, and if the elementary matrix E results by 
performing a certain row operation on the identity matrix, then the product EA is the 
matrix that results when the same row operation is performed on A. 

In short, this theorem states that an elementary row operation can be perfonned on a 
matrix A using a left multiplication by and appropriate elementary matrix. 


Example 6 (Performing Row Operations by Matrix Multiplication). Consider the 


matrix 



0 2 3 

-13 6 
4 4 0 


Find an elementary matrix E such that EA is the matrix that results by adding 4 times the 
first row of A to the third row. 


Solution: The matrix E must be 3x 3 to conform to the product EA. Thus, we obtain E 

"1 0 0 " 

by adding 4 times the first row of / 3 to the third row. This gives usE = 0 1 0 

4 0 1 


"1 

0 

0 " 

"1 

0 

2 

3l 


'1 

0 

2 

3" 

0 

1 

0 

2 

-1 

3 

6 

- 

2 

-1 

3 

6 

4 

0 

1 

1 

4 

4 

0 


5 

4 

12 

12 


As a check, the product EA is EA = 

So left multiplication by E does, in fact, add 4 times the first row of A to the third row. 


If an elementary row operation is applied to an identity matrix / to produce an elementary 
matrix E, then there is a second row operation that, when applied to E, produces I back 
again. 


For example, if E is obtained by multiplying the i-th row of / by a nonzero scalar c, then 
I can be recovered by multiplying the i-th row of E by 1/c. The following table explains 
how to recover the identity matrix from an elementary matrix for each of the three 
elementary row operations. The operations on the right side of this table are called the 
inverse operations of the corresponding operations on the left side. 


Row operation on I that produces E 

Row operation on E that reproduces I 

Multiply row i by c ^ 0 

Multiply row i by 1/c 

Interchange rows i and j 

Interchange rows i and j 

Add c times row i to row j 

Add -c times row i to row j 
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Example 7 (Recovering Identity Matrices from Elementary Matrices). Here are three 
examples that use inverses of row operations to recover the identity matrix from 


“1 

o' 


“1 O' 


"1 

O' 



—» 


—» 



0 

1 

A A 

O 7_ 


0 

1 


the second 
row by 7. 


the second 
row by 1/7. 


1 

0 


0 

1 


1 

O' 

0 

1 

Interchange 

1 

0 

Interchange 

0 

1 


the first and 
second rows. 


the first and 
second rows. 


‘1 

o' 

—» 

'1 

5 

—» 

'1 

0" 

0 

1 


0 

1 


0 

1 


Add 5 times 
the second row 
to the first. 


Add -5 times 
the second row 
to the first. 


The next theorem is the basic result on the invertibility of elementary matrices. 


Theorem An elementary matrix is invertible and the inverse is also an elementary 
matrix. 


Example 8 



' 1 

0 

O' 


"0 

1 

o' 


'1 

0 

0 " 

Let E x = 

0 

1 

0 

f - 

9 -^2 

1 

0 

0 

f - 

9 ^3 

0 

1 

0 


-4 

0 

1 


0 

0 

1 


0 

0 

5 


a b c 


A = 


d e f 


g h i 


Compute EiA, E 2 A, E 3 A and describe how these products can be obtained by elementary 
row operations on A. 

Solution We have 



a b c 


~d 

e 

f 


a 

b 

c 

E X A = 

d e f 

, e 2 a = 

a 

b 

c 

,e 3 a = 

d 

e 

f 


g - 4a h-4b i- 4c 


_g 

h 

i 


_ 5 & 

5 h 

5 i 
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Addition of (-4) times row 1 of A to row 3 produces EjA. (This is a row replacement 
operation.) An interchange of rows 1 and 2 of A produces E 2 A and multiplication of row 
3 of A by 5 produces E$A . 

Left-multiplication (that is, multiplication on the left) by E] in Example 8 has the same 
effect on any 3 xn matrix. It adds - 4 times row 1 to row 3. In particular, since Ei I = 
E/, we see that Ej itself is produced by the same row operation on the identity. Thus 
Example 8 illustrates the following general fact about elementary matrices. 


Note: Since row operations are reversible, elementary matrices are invertible, for if E is 
produced by a row operation on 7, then there is another row operation of the same type 
that changes E back into 7. Hence there is an elementary matrix F such that FE = 7. Since 
E and F correspond to reverse operations, EF = I. 


Each elementary matrix E is invertible. The inverse of E is the elementary matrix of the 
same type that transforms E back into 7. 


Example Find the inverse of E x 


1 0 0 
0 1 0 
-4 0 1 


Solution: To transform Ej into 7, add + 4 times row 


The elementary matrix which does that is E x 1 


1 

0 


+4 


to row 3. 
0 0 " 

1 0 
0 1 


Theorem An nxn matrix A is invertible if and only if A is row equivalent to 7 n , and in 
this case, any sequence of elementary row operations that reduces A to /„ also transforms 
I a into A' 1 . 


An Algorithm for Finding A' 1 If we place A and 7 side-by-side to form an augmented 
matrix [A 7], then row operations on this matrix produce identical operations on A and 7. 
Then either there are row operations that transform A to 7„, and 7„ to A' 1 , or else A is not 
invertible. 


Algorithm for Finding A' 1 

Row reduce the augmented matrix [A I\. If A is row equivalent to 7, then [A 7] is 
row equivalent to [7 A ' 1 ]. Otherwise, A does not have an inverse. 


©Virtual University Of Pakistan 


136 





12- Inverse of a Matrix 


VU 


0 1 2 

1 0 3 , if it exists. 

4-3 8 

10 3 0 1 0" 

) 1 2 10 0 R n 

1 -3 8 0 0 1 

— 4f?j + R 3 3 R-, + R^ 

"1 0 3 0 1 0] fl 0 3 0 1 0" 

~0 1 2 100-012100 
0 -3 -4 0 -4 lj [o 0 2 3 -4 1 


—2 R^ + R-, — 3Rj + 


0 

3 

0 

1 

0 ' 


"l 

0 

0 

-9/2 

7 

-3/2 

1 

2 

1 

0 

0 

- 

0 

1 

0 

-2 

4 

-1 

0 

1 

3/2 

-2 

1/2 


0 

0 

1 

3/2 

-2 

1/2 


-9/2 7 -3/2 

Since A -1, we conclude that ,4 is invertible, and A~ l - -2 4 -1 

3/2 -2 1/2 

It is a good idea to check the final answer: 

"0 1 21T-9/2 7 -3/2] [1 0 0" 

d/f 1 = 1 0 3 -2 4 -1 = 0 1 0 

4 -3 8J |_ 3/2 -2 1/2 J [o 0 1 

It is not necessary to check that A' 1 A = I since A is invertible. 

1 2-3 1 

-1 3 -3 -2 

, if it exists. 

2 0 15 

3 1-25 



Example 9 Find the inverse of the matrix A = 

"0 1 2 10 0 ]' 

Solution [A I] = 1 0 3 0 1 0 - 

4 -3 8 0 0 1 


©Virtual University Of Pakistan 


137 




12- Inverse of a Matrix 


VU 


Consider detA = 


1 

2 

-3 

1 


1 

2 

-3 1 

-1 

3 

-3 

-2 


0 

5 

-6 -1 

2 

0 

1 

5 


0 

-4 

7 3 

3 

1 

-2 

5 


0 

-5 

7 2 


operating + ^3 ’ ^1 ’ ^4 ’ ^^1 


Expand from first column = 

As the given matrix is singular, so it is not invertible. 


5 

-6 

-1 


5 

-6 

-1 

-4 

7 

3 

= 

1 

1 

2 

-5 

7 

2 


0 

1 

1 


= 5( 1-2)+ 6(1-0)-1(1-0) = 0 


Example 11 Find the inverse of the given matrix if possible A = 


1 

2 

3 


0 1 
1 1 
1 1 


Solution 


det A = 


1 

2 

3 


0 1 
1 1 
1 1 


-1 


As the given matrix is non-singular therefore, inverse is possible. 


1 

O 

_ 1 


0 

0 

1 _ 

2 1 1 


0 1 0 

3 1 1 


0 0 1 


1 

0 

_ 1 


0 

0 

1 _ 

0 1 -1 


-2 1 0 

0 1 -2 


-3 0 1 


R 2 - 2R X , R 3 - 37?, 
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"10 1 1 


o 

o 

i_ 

0 1 -1 


-2 1 0 

0 0-1 


-1 -1 1 


i? 3 R 2 


"1 

0 

1' 


' 1 

0 

0" 

0 

1 

-1 


-2 

1 

0 

0 

0 

1 


1 

1 

-1 


MultiplyR, by -1 


1 

O 

O 

i 


"o -i r 

0 1 0 


-1 2 -1 

0 0 1 


1 1 -1 


, R n + i? 3 


Hence the inverse of matrix A is A 1 


0 -1 1 

-1 2 -1 

1 1 -1 


Example 12 Find the inverse of the matrix A = 


1 2 2 
2 2 3 
5 2 3 


Solution 


det A = 


1 2 2 
2 2 3 
5 2 3 


= 6 


As the given matrix is non-singular, therefore, inverse of the matrix is possible. 
We reduce it to reduce echelon form. 


"1 2 2] 


o 

o 

i_ 

2 2 3 


0 1 0 

5 2 3 


0 0 1 
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"12 21 


' 1 

0 O' 



0 -2 -1 


-2 

1 0 



0 -8 -7J 


-5 

0 1 



R 2 -2R l ,R 3 - 

57?! 





"12 2 

1 

' 1 

0 

O' 


0 1 1/2 


1 

-1/2 

0 


0 -8 -7 


-5 

0 

1 


multiply 2nd 

row by -1/2 





"12 2 


"1 

0 

O' 


0 1 1/2 


1 

-1/2 

0 


0 0-3 


3 

-4 

1 


7? 3 + 8 R 1 





"12 2 


' 1 

0 


0 

0 1 1/2 


1 

-1/2 


0 

0 0 1 


-1 

4/3 

- 

-1/3 


Multiply 3rd row by -1/3 



" 3 _8 2 ' 

"1 2 O' 


3 3 

0 1 0 


3/2 -7/6 1/6 

0 0 1 


-1 4/3 -1/3 

"1 0 O' 


" 0 -1/3 1/3 " 

0 1 0 


3/2 -7/6 1/6 

0 0 1 


-1 4/3 -1/3 


7? 2 -(l/2)7? 3 ,7? 1 -27? 3 


Hence the inverse of the original matrix A 1 


0 -1/3 1/3 

3/2 -7/6 1/6 

-1 4/3 -1/3 
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Exercises 

In exercises 1 to 4, find the inverses of the matrices, if they exist. Use elementary row 
operations. 








'1 

0 

5“ 



'1 

2] 








1. 

5 

9 



2. 

1 

1 

0 








3 

2 

6 



“ 1 


4 - 

-3l 


“ 1 

3 

- 

-fi 

3. 

-2 

- 

-7 

6 

4. 

0 

1 


2 


1 


7 - 

-2 


-1 

0 

8j 


"3 

4 

-ll 



'-1 

3 

- 

-4“ 

5. 

1 

0 

3 


6. 

2 

4 


1 


2 

5 

-4 



-4 

2 

- 

-9 


7. Let A = 


columns. 


-1 -5 -7 
2 5 6 

1 3 4 


. Find the third column of A' 1 without computing the other 


-25 -9 -27 

8. Let A = 546 180 537 
154 50 149 

computing the first column. 


. Find the second and third columns of A' 1 without 


9. Find an elementary matrix E that satisfies the equation. 


(a) EA = B 

(b) EB = 

A 

(c) EA = C 

(d) EC = A 


"3 4 1 1 


"8 1 5 1 


“34 r 

where A = 

2 -7 -1 

,B = 

2 -7 -1 

,c = 

2 -7 -1 


8 1 5 


3 4 1 


2-7 3 


10. Consider the matrix A = 


1 0 
0 1 
0 0 


-2 

0 

2 


(a) Find elementary matrices Ei and E 2 such that E 2 EiA=I. 

(b) Write A' 1 as a product of two elementary matrices. 
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(c) Write A as a product of two elementary matrices. 

In exercises 11 and 12, express A and A’ 1 as products of elementary matrices. 



'2 

1 

f 


'1 

1 

0 " 

11. A = 

1 

2 

1 

12. A = 

1 

1 

1 


1 

1 

2 


0 

1 

1 


0 17 8 

13. Factor the matrix A = 1 3 3 8 as A = EFGR, where E, F, and G are 

-2 -5 1 -8 

elementary matrices and R is in row echelon form. 
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Lecture 13 

Characterizations of Invertible Matrices 

This chapter involves a few techniques of solving the system of n linear equations in n 
unknowns and transformation associated with a matrix. 

Solving Linear Systems by Matrix Inversion 
Theorem 

Let A be an n x n invertible matrix. For any beR" , the equation^* = b .(1) has the 

unique solution i.e. x =A' 1 b. 

Proof 

Since A is invertible and b e R" be any vector. Then, we must have a matrix A 'b which 
is a solution of eq. (1) .i.e. Ax = A (A ~ r b) = lb = b. 

Uniqueness 

For uniqueness, we assume that there is another solution u. Indeed, it is a solution of 
eq.( 1) so it must be u = A^b, it means x = A~ ] b = u. This shows that u=x. 

Theorem 

Let A and B be the square matrices such that AB = I. Then, A and B are invertible with 
B = A' 1 and A = B' 1 

Example 1 

Solve the system of linear equations 

2xj + x 2 + x 3 =1 
5x, + x 2 + 3x 3 = 3 
Xj + 4 x 3 = 6 

Solution 

Consider the linear system 

2X[ + x 2 + x 3 =1 
5Xj + x, + 3 x 3 = 3 
Xj + 4 x 3 = 6 

The Matrix form of system is Ax = b, where 
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~ 2 1 11 r^i rr 

A = 5 1 3, x = x 2 , b = 3 

10 4 x 3 6 


Here, det (A) = 2(4) - 1 (20 - 3) + 1(0 - 1) = 8 - 17 - 1 = -10* 0 
So, A is invertible. Now, we apply the inversion algorithm: 


2 11 10 0 

5 13 0 10 

1 0 4 0 0 1 


1 0 
0 1 

0 0 


1 0 4 0 0 1 

5 13 0 10 

2 11 10 0 



'1 

0 

4 


0 

0 

1 


0 

1 - 

-17 


0 

1 

_ 


0 

1 

-7 


1 

0 


"1 

0 

4 


0 

0 


1 ' 

0 

1 

-17 


0 

1 


-5 

0 

0 

10 


1 

-1 


3 


4 

-17 

1 


0 0 1 
0 1 -5 

j_ -1 
10 10 10 


10 


— 5 + R 7 , 


R 2 +R 3 , 


— 27?j + R 2 
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Hence, A 1 


-2 

2 

-l 

5 

5 

5 

17 

-7 

1 

To 

To 

To 

l 

-l 

3 

To 

To 

10 


Thus, the solution of the linear system is x = A l b = A 1 


"-2 

2 

-f 



'-2' 

T 

5 

T 

'1] 


5 

17 

-7 

l 


1 



— 

3 

= 

— 

To 

To 

10 

6 


5 

l 

-l 

3 



8 

_To 

To 

10. 



_ 5 _ 


Example # 2 Solve the system 


x l + 2x, + 3x 3 = 5 


2x x + 5x 2 + 3x 3 =3 by inversion method. 


Xj + 


8x 3 = 17 


Solution Consider the linear system 


Xj + 2 x 2 + 3x 3 = 5 
2xj + 5 x 2 + 3x 3 = 3 


Xj + 


8x 3 = 17 


This system can be written in matrix form as Ax = b, where 


"1 2 3] 




“5“ 

2 5 3 

, x = 

x 2 

, b = 

3 

1 0 8J 


_x 3 J 


17 


Here, [det (A)\ = 40 - 2 (16 - 3) + 3(0 - 5) = 40 - 26 - 15 = -1 * 0 
Therefore, A is invertible. 


Now, we apply the inversion algorithm: 


1 2 3 

2 5 3 
1 0 8 


1 0 0 
0 1 0 
0 0 1 


1 2 3 

0 1 -3 

0-2 5 


1 0 

-2 1 

-1 0 


—2i?j + R 2 , 


-\R t +R 3 
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Hence, 


A~ l = 


2 

1 

0 

2 

1 

0 

2 

1 

0 


3 

-3 

-1 

3 

-3 

1 

0 

0 

1 


0 0 
1 0 
0 1 
-40 
13 
5 


1 

-2 
-5 

1 

-2 
5 

1-14 
13 
5 

|-40 
13 
5 

16 9 

-5 -3 
-2 -1 


0 0 
1 0 
2 1 
0 0 


2R 2 + R 3 


1 

-2 

6 

-5 

-2 

16 

-5 

-2 


0 

-1 

3 ' 
-3 

-1 

9 

-3 

-1 


-1 R, 


3 R 3 + R *,, 


-2R 2 + R j 


—3i?3 + /?] 


Thus, the solution of the linear system is x = A 'b = 


-40 

16 

9 " 

“5“ 


' 1 ' 

13 

-5 

-3 

3 

= 

-1 

5 

-2 

-1 

17 


2 


ThusXj =l,x 2 = -l,x 3 = 2 . 


Note: It is only applicable when the number of equations = number of unknown and fails 
if given matrix is not invertible. 

Example 3 

Solve the system of linear equation 

x l + 6 x 2 + 4x 3 = 2 
2Xj + 4x 2 - x 3 = 3 
—Xj + 2 x 2 + 5x 3 = 3 


Solution 

The matrix coefficient 


A = 


1 6 
2 4 

-1 2 


4 
-1 

5 


det (A) = 1(20 + 2) - 6(10 - 1) + 4 (4+4) 
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= 22-54 + 32 
= 0 

Thus, A is not invertible. Hence, the inversion method fails. 

Solution for the system of a n x n Homogeneous linear equations with 
an Invertible Coefficient Matrix:- 

Let see if the system is considered as homogeneous then what does the above theorem 
say? 

Theorem:- 

Let Ax = 0 be a homogeneous linear system of n equations and n unknowns. Then, the 
coefficient matrix A is invertible iff this system has only a trivial solution. 

Example 4 

State whether the following system of linear equation has a solution or not? 

2xj + x 2 + 3 x 3 = 0 
x { - 3 x 2 + x 3 = 0 
v, - 4x 3 = 0 


Solution 

We see 


A = 


2 

1 

1 


1 

-3 

0 


3 

1 

-4 


is an invertible matrix (det (A) + 0 ) 


Thus, this homogeneous linear system has only the trivial solution. 


Example 5 


Solve < 


Xj + x 2 + x 3 = 8 
2x 2 + 3 x 3 = 24 
5xj + 5 x 2 + x 3 = 8 


Solution This system can be written in matrix form as Ax = b, where 


“l 1 ll 




“8 " 

0 2 3 

, x = 

x 2 

, b = 

24 

5 5 1 


x 3 


8 
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Here [det (A)\ = 1(2 - 15) - 1(0 - 15) + 1(0 - 10) = -13 + 15 - 10 = -8 ± 0 
Therefore, A is invertible. 

Now, we apply the inversion algorithm: 

A I 3 

"111 100 " 

0 2 3 0 1 0 

5 5 1 0 0 1 

fl 1 1 I 1 00] 


0 2 3 0 1 0 -5R\ + R 3 

0 0 -4 -5 0 1 
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Hence, 





Thus Xj = 0, x 2 = 0, x 3 = 8. 


Theorem (Invertible Matrix Theorem) Let A be a square n x n matrix. Then the 
following statements are equivalent. (Means if any one holds then all are true). 

(a) A is an invertible matrix. 

(b) A is row equivalent to the n x n identity matrix. 

(c) A has n pivot positions. 

(d) The equation Ax = 0 has only the trivial solution. 

(e) The columns of A form a linearly independent set. 

(f) The linear transformation x —> Ax is one-to-one. 

(g) The equation Ax = b has at least one solution for each b in if". 

(h) The columns of A span if". 

(i) The linear transformation x —> Ax maps if" onto if". 

(j) There is a n x n matrix C such that CA = I. 

(k) There is a nxn matrix D such that AD = I. 

(l) A t is an invertible matrix. 


Example 6 


1 0 

Show that the matrix A = 1 1 

0 1 


-4 

5 is invertible by using Invertible Matrix Theorem 
2 
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Solution 

By row equivalent, 

"1 0 -4 

A = 1 1 5 77, 

0 1 2 
1 0 -4~ 

0 1 9 , — R ? + 77, 

0 1 2 
1 0 -4 
0 1 9 

0 0-7 

It shows that A has three pivot positions and hence is invertible, by the Invertible Matrix 

Theorem (c). 


Example 7 


Use the Invertible Matrix Theorem to decide if A is invertible, where 
"1 0 -2 
A= 3 1-2 

-5 -1 9 

Solution 

"1 0 - 2 ] [1 0 -2 

A = 0 1 4 ~ 0 1 4 Here, A has three pivot positions and hence is 

0 -1 -lj [o 0 3 

invertible by the Invertible Matrix Theorem (c). 


Example 7 Find A 1 and show that A 1 is an invertible matrix. 

12 10 

2 4 11 
A = 

10 11 
0 111 

Solution 


"12 10 " 

, 2 4 0 1 

A‘ = 

1111 
0 111 
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Now, by row equivalent of A, 


R 2 -2R t ,R, - R t R 23 


"1 

2 

1 

0 ] 


“1 

2 

1 

0 ~ 


“1 

2 

1 

0 " 

2 

4 

1 

1 


0 

0 

-1 

1 


0 

-2 

0 

1 

1 

0 

1 

1 


0 

-2 

0 

1 


0 

0 

-1 

1 

0 

1 

1 

1 


0 

1 

1 

1 


0 

1 

1 

1 


2 ^2 R4 ^2 ( 1)^3 ^3 + ^4 




1 

_ 


“l 

2 

1 

0 " 


“1 

2 

1 

0 ~ 


“1 

2 

1 

0 " 

1 

2 

0 
















1 





1 





-1 





-1 

0 

1 

0 


0 

1 

0 



0 

1 

0 



0 

1 

0 







2 





2 





2 

0 

0 

-1 

Z 


0 

0 

-1 

1 


0 

0 

1 

-1 


0 

0 

1 

-1 

l 



3 



3 




0 

1 

1 

l 


0 

0 

1 


0 

0 

1 


0 

0 

0 

5 









2 _ 





2 _ 





2 _ 


Here A has 4 pivot positions so by Invertible Matrix Theorem (c) A is invertible. Thus, 
by ( 1 ) A' is invertible. 

Example 8 

Use the Invertible Matrix Theorem to decide if A is invertible, where 
12 10 

2 4 12 
A = 

10 12 
0 112 

Solution 






R2 

-2R l 




R 22 



"1 

2 

1 

0 ~ 


"1 

2 

1 

0 ~ 


“1 

2 

1 

0 " 

2 

4 

1 

2 


0 

0 

-1 

2 


0 

-2 

0 

2 

1 

0 

1 

2 


0 

-2 

0 

2 


0 

0 

-1 

2 

0 

1 

1 

2 


0 

1 

1 

2 


0 

1 

1 

2 
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- — R 4 -R 2 _ ^3 + J ^4 


"1 

2 

1 

o ' 


'1 

2 

1 

o ' 


'1 

2 

1 

0 ' 


'1 

2 

1 

0 " 

0 

1 

0 

-1 


0 

1 

0 

-1 


0 

1 

0 

-1 


0 

1 

0 

-1 

0 

0 

-1 

2 


0 

0 

-1 

2 


0 

0 

1 

-2 


0 

0 

1 

-2 

0 

1 

1 

2 


0 

0 

1 

3 _ 


0 

0 

1 

3 _ 


0 

0 

0 

5 _ 


Here, A has 4 pivot positions and hence is invertible by Invertible Matrix Theorem (c). 


Solving Multiple Linear Systems with a Common Coefficient Matrix 

This technique is used in solving a sequence of linear systems 


Ax j = hj, Ax 2 =b 2 , • • • , Ax k =b k (1) 

where coefficient matrix A remains same and off course if it is invertible, then we have a 
sequence of solutions, i.e. 
x, = A~'b ] , x 2 =A l b 2 , ••• , x k =A~ l b k . 

Find a Matrix from Linear Transformation 

We can find a matrix corresponding to every transformation. In this section we will learn 
how to find a matrix attached with a linear transfonnation. 

Example 9 

Let L be the linear transformation from R to P 2 (Polynomials of order 2) defined by 
T(x, y) = x y t + (x + y)t 2 

Find the matrix representing T with respect to the standard bases. 

Solution 

LetA= {(1,0),(0,1)} be the basis of R 2 , then 

T(1,0) = t = (0,0,1) (This triple represents the coefficients of polynomial t ) 
i.e. t = 0.1 + O.t +1 ,t~ 


Similarly, T (0, 1) = t 2 = (0,0,1). Hence, the matrix is given by 

r o O' 


A= 


0 0 

d 1, 


Now, we will proceed with a more complicated example. 
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Example 10 

Let T be the linear transformation from R to R such that T(x, y) = (x , y + 2x). Find a 
matrix A for T. 

Solution 

This matrix is found by finding T (1, 0) = (1, 2) and T (0, 1) = (0, l)The matrix 

f 1 

is A = 

U ij 

Important Note 

It should be clear that the Invertible Matrix Theorem applies only to square matrices. For 
example, if the columns of a 4x3 matrix are linearly independent, we cannot use the 
Invertible Matrix Theorem to conclude any thing about the existence or nonexistence of 
solutions to equations of the former = b. 

Invertible Linear Transformations 

Recall that matrix multiplication corresponds to composition of linear transformations. 
When a matrix A is invertible, the equation A' 1 Ax = x can be viewed as a statement about 
linear transformations. See Figure 2. 


Multiplication 



Figure 2 A' 1 transfonns Ax back to x 


Definition 

A linear transformation T : R" —> R" be a linear Transfonnation and A be a standard 
matrix for T. Then, T is invertible if and only if A is an invertible matrix in that case 
linear transformation S given by S(x) = A -I x is a unique function satisfying (1) and (2) 

S{T{x)) = x V xeR" (1) 

T(S(x)) = xV xeR" (2) 
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Important Note 

If the inverse of a linear transformation exits then it is unique. 

Proposition 

Let T : R" —> R'“ be linear transformation, given as T(x) = Ax ,Vxe R“ , where A is 
a m x n matrix. The mapping T is invertible if the system y = Ax has a unique solution. 
Case 1: 

If m < n , then the system Ax = y has either no solution or infinitely many solutions, for 
any y in R m Therefore, y = Ax is non-invertible. 

Case 2: 

If in = n , then the system Ax = y has a unique solution if and only if Rank (A) = n. 

Case 3: 

If m > n , then the transfonnation y = Ax is non-invertible because we can find a vector 
y in R m such that Ax = y is inconsistent. 
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Exercises 


1.Solve the system of linear equations by inverse matrix method. 

Xj + x 2 + 4x 3 = 2 
2x 2 + 3x 3 = 4 
5xj + x 2 - x 3 =3 



'1 2' 


“5“ 


"-2] 


"3l 


V 

2. Let A = 


,b, = 


,b, = 


,b 3 = 


, and b. = 



1 

oo 
1_ 

7 I 

7 

7 Z 

-2 


_5J 

4 

7 


(a) Find A' 1 and use it to solve the equations Ax = bi, Ax = b 2 , Ax = b 3 , Ax = b 4 . 

(b) Solve the four equations in part (a) by row reducing the augmented matrix 

[ A ^ b 2 h K]- 

3. (a) Solve the two systems of linear equations 

Xj + 2 x 2 + x 3 = -1 
Xj + 3 x 2 + 2x 3 = 3 
x 2 + 2x 3 = 4 

& 

Xj + 2 x 2 + x 3 = 0 
Xj + 3x 2 + 2x 3 = 0 
x 2 + 2x 3 = 4 

by row reduction. 

(b) Write the systems in (a) as Ax = bi and Ax = b 2 , and then solve each of them by the 
method of inversion. 

Determine which of the matrices in exercises 4 to 10 are invertible? 


4. 

1 

SO 

^J" 

1 

1_ 

5. 

"5 0 3] 
7 0 2 

6. 

'2 3 4" 

2 3 4 

7. 

"5 -9 3" 

0 3 4 


L 3 -9 


9 0 1 


2 3 4 


1 0 3 
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"13 0-1] [1 0 0 0] [7-6-41 

0 1 -2 -1 2 5 0 0 -5 1 0 -2 

8. 9. 10. 

-2 -6 3 2 3 6 8 0 10 11 7 -3 

_ 3 5 8 -3J ^4 7 9 loj |_ 19 9 7 1 

'5 4 3 6 3 

7 6 5 9 5 

11. 8 6 4 10 4 

9 -8 9-5 8 

10 8 7 -9 7 

12. Suppose that A and B are n x n matrices and the equation ABx = 0 has a nontrivial 
solution. What can you say about the matrix AB1 

13. What can we say about a one-to-one linear transfonnation T from R" into if"? 

14. Let T : R 2 —» R 2 be a linear transformation given as T (x) = 5x , then find a matrix A of 
linear transfonnation T. 

In exercises 15 and 16, T is a linear transformation from R into R . Show that T is 
invertible. 

15. T(x l ,x 2 ) = (-5x 1 +9x 2 ,4x l -7x 2 ) 

16. T(x x ,x 2 ) = (6x 1 -8x 2 ,-5xj +7x 2 ) 

17. Let T : R" —> R" be a linear transformation and let A be the standard matrix for T. 
Then T is invertible if and only if A is an invertible matrix. 
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Lecture 14 

Partitioned Matrices 

A block matrix or a partitioned matrix is a partition of a matrix into rectangular 
smaller matrices called blocks.Partitioned matrices appear often in modem applications 
of linear algebra because the notation simplifies many discussions and highlights 
essential structure in matrix calculations. This section provides an opportunity to review 
matrix algebra and use of the Invertible Matrix Theorem. 


General Partitioning of a Matrix - 

A matrix can be partitioned (subdivided) into sub matrices (also called blocks) in 
various ways by inserting lines between selected rows and columns. 

Example 1 

The matrix 


P = 


112 2 
112 2 
3 3 4 4 
3 3 4 4 


can be partitioned into four 2x2 blocks 



fl 1 



2^ 


^3 

3^ 


M 4^ 

Pu=\ 

vl 1 ) 

p = 

’M2 

v2 

2; 

P = 

21 

v3 

3; 

P = 

’ 1 22 

v4 4 y 


The partitioned matrix can then be written as 



V J 21 1 22 J 



can also be written as the 2x3 partitioned (or block) matrix 
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A = 



whose entries are the blocks or sub matrices. 

Note:- 

It is important to know that in how many ways to block up an ordinary matrix A. See the 
following example in which a matrix A is block up into three different ways. 

Example 3 

Let A be a general matrix of 5 x 3 order, we have 

Partition (a) 


2 CL 12 $13 


$2i ^^22 $23 




$41 $42 ^43 


In this case we partitioned the matrix into four sub matrices. Also notice that we 
simplified the matrix into a more compact form and in this compact fonn we’ve mixed 
and matched some of our notation. The partitioned matrix can be thought of as a smaller 
matrix with four entries, except this time each of the entries are matrices instead of 
numbers and so we used capital letters to represent the entries and subscripted each one 
with the location in the partitioned matrix. 

Be careful not to confuse the location subscripts on each of the sub matrices with the size 

of each sub matrix. In this case An is a 2 x 1 sub matrix of A, An is a2 x 2 sub matrix 
of A, A?i is a 3x1 sub matrix of A and A 22 is a 3x3 sub matrix of A. 
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Partition (b) 



In this case, we partitioned A into three column matrices each representing one column in 
the original matrix. Again, note that we used the standard column matrix notation (the 
bold face letters) and subscripted each one with the location in the partitioned matrix. 

The c; in the partitioned matrix are sometimes called the column matrices of A. 

Partition (c) 


A = 


r 




a \\ 

a \2 

a n 


r x 

a 2\ 

a 22 

a 23 


r 2 

a 2l 

a i2 

a 33 

= 

>3 

«41 

a 42 

a 43 


r 4 

IT) 

o | 

a 52 

a 53 


|/sj 


Just as we can partition a matrix into each of its columns as we did in the previous part 
we can also partition a matrix into each of its rows. The r, in the partitioned matrix are 
sometimes called the row matrices of A. 

Addition of Blocked Matrices 

If matrices A and B are the same size and are partitioned in exactly the same way, then it 
is natural to make the same partition of the ordinary matrix sum A + B. In this case, each 
block of ^4 + B is the (matrix) sum of the corresponding blocks of A and B. 

Theorem 

If A is mx n and B is nx p, then 
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AB = Col x (A) + Col 2 (A) + ... + Col n (A) 


Row x ( B ) 
Row 2 ( B ) 


Row n (B ) 


= CoI l ( A)Row l (B) + Col 2 (A)Row 2 (B) + ... + Col n (A)Row n (B) 


Proof 

For each row index I and column index j, the (i,j) entry in cofi (A) rowk (B) is the product 
of a;k from colk (A) and bkj from rowk (B). 

Hence, the (i,j)-entry in the sum shown in (2) is 


a ,Aj +a 2i b 2j +- + a „A J 

(k = 1,2 


This sum is also the (i,j)- entry in AB by the row column rule 


Example 4 

Let 


A = 


A 2 

and B - 

~B n 



_Ai 

A 2 _ 


b 2 1 

1 

CN 
_ <N 


A + B - 

~A , 

A 2 

+ 

~ B n 

B \2 


_Ai 

A 22 _ 


_ B 2l 

B 22 _ 


A n +B l{ A n + B n 
_A 2 \ t B 2l A 22 + B 22 ~ 

Similarly, subtraction of blocked matrices is defined. 

Multiplication of a partitioned matrix by a scalar is also computed block by block. 

Multiplication of Partitioned Matrices 




~A l 

A 2 



\B U 

b.A 

If 

A = 

Ai 

A2 

and 

B = 

11 

B„ 

12 

B-n 



Ai 

A2 _ 



L 21 

22 J 


and if the sizes of the blocks confirm for the required operations, then 
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AB = 


21 


12 


22 


^31 ^32 


ft, B, 


12 


B -,, f? 


21 


22 . 


^lAn ^ 12^21 

AAn ^22^21 

AAn ^32^21 


A\\B\2 + '^ 12^22 
AA2+A2B22 
AAu ‘AA 22 


It is known as block multiplication. 

Example 5 

Find the block multiplication of the following partitioned matrices: 


Solution 



" 1 

2 

1 

0 

2 

A = 

-1 

-2 

3 

1 

1 


2 

1 

-2 

1 

3 


X 21 


-a 2 


x 22 


5 = 


“ 2 

f 

3 

0 

-5 

1 

0 

8 

0 

2 




B 


21. 


Let 


4,= 


*„ = 


Now 


1 

2 

ll 

, A, n = 

ro 

2 ^ 

-1 

-2 

3 

9 12 

vl 

b 


2 

3 


-5 1 


A 2i = [2 1 -2] and A 22 = [l 3]: 


’ ^21 — 


0 8 
0 2 


So B = 


B 1 
5 


21. 


AB = 

1 

<N 

1_ 

~b x r 


AlAl + -^12-^21 


1 

<N 

_ B 2\ _ 


A 2X B n + A 22 B 2l 


This is a valid formula because the sizes of the blocks are such that all of the operations 
can be perfonned: 


A\\B\\ + AB 21 




+ 


0 2 

vl b 


8 

2 


A\B\\ + A 2 B 21 

= [17 0] + [0 14] = 


" 2 

f 







'0 

8" 

3 

0 

+[1 4 

0 

2 

-5 

1 





[17 14] 


3 4 

-23 14 


Thus, 
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" 3 

6" 

A 1^11 + -^12-^21 


-23 

12 

_ A1 1 B^ | + A 22 B 2X _ 


17 

14 


Note - We see result is same when we multiply A and B without partitions 



“ 2 f 



" 1 2 10 2“ 

3 0 


' 3 6“ 

AB = 

-1-2 3 11 

-5 1 

= 

-23 12 


2 1-213 

0 8 


17 14 


0 2 



Note - Sometimes it is more useful to find the square and cube powers of a matrix. 

Example 6 


The block version of the row -column rule for the product AB of the partitioned matrices 



“3-4 1 

0 2“ 






A u A„ 

A = 

-1 5 -3 

1 4 

— 

11 12 





A n . A„ 


2 0-2 

1 6 


L / 1 ^22 J 


where 


2 

-f 

3 

0 

-5 

1 

4 

-3 

0 

2 


B u 

_ B 2\ _ 


B n 


2 

3 

-5 


-1 

0 

1 


and B 2l 


'4 -3" 
vO 


So 


,45 = 

"4i 

1 

(N 

B n ~ 


A\\B n + d l2 B 2l 


_^21 

K > 

1_ 

B n _ 


_Ai\B n + A 22 B 2l _ 


This is a valid formula because the sizes of the blocks are such that all of the operations 
can be perfonned: 


^]\B\ \ + A I2 B 21 



“ 2 

-l 









“3 -4 r 

3 

0 

+ 

“0 

2“ 

“4 

-3“ 

_ 

"-11 

2“ 

5 -3_ 


1 


1 

4 

0 

2 


32 

3_ 


-5 










A 2I B n + A 22 B 21 - [2 0 -2] 


" 2 

-f 






'4 -3" 

3 

0 

+ [1 6 ] 

0 2 

-5 

1 




[18 5] 


Thus, 
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"-11 2" 


A ,, B ,, H" ^4, 9 ! 



AB = 

11 11 12 21 

= 

32 3 


^ 21^11 + A 22 B 21 _ 


18 5 


This result can be confirmed by performing the computation. 



“ 2 -1“ 



'3 -4 1 0 2“ 

3 0 


“-11 2“ 

AB = 

-15-314 

-5 1 

= 

32 3 


2 0-216 

4 -3 


18 5 


0 2 



Example 7 

Making block up of matrix 


A = 


2 0 0 0 
0 2 0 0 
0 0 2 0 
0 0 0 2 
0 0 0 0 


2 

2 

2 

0 

2 


evaluated ? 


Solution 


We partition A as shown below 


Now 


A = 


2 0 0 

0 2“ 



0 2 0 

0 2 


“0 2“ 

0 0 2 

0 2 

where A n = 

0 2 

0 0 0 

0 0 0 

2 0 

0 2_ 


0 2 


A 2 = 


2^3 ^32 


V ^23 


21 


2 J 


2^3 ^32 

cl 2/, 


V^23 


2 J 


" 4 ■/, 4 V 

v 0 2 3 4/ 2 y 
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Hence 



~4 

0 

0 

0 

8] 




0 

4 

0 

0 

8 



A 2 = 

0 

0 

4 

0 

8 




0 

0 

0 

4 

0 




0 

0 

0 

0 

4 



Example 8 








'1 

0 

0 

0 

0 

1 



0 

1 

0 

0 

0 

1 



0 

0 

1 

0 

0 

1 

Let A = 









0 

0 

0 

1 

0 

0 



0 

0 

0 

0 

1 

0 



1 

1 

1 

0 

0 

1 


Evaluate A 2 ? 


Solution 


We partition A as shown below 


1 

0 

0 

0 

0 

f 

0 

1 

0 

0 

0 

1 

0 

0 

1 

0 

0 

1 

0 

0 

0 

1 

0 

0 

0 

0 

0 

0 

1 

0 

1 

1 

1 

0 

0 

1 


h 

Q2 

A 1 


T 

^23 

h 

021 

where A x = 

1 

A 

0\2 

1 


1 




h 

^32 

A " 

A 

(N 

cT 

A 1 


/ 3 + A X A‘ 

O 32 

A + A 

Now 

A 2 = 

^23 

h 

0 2l 

O 23 

h 

0 2 1 

= 

^23 

h 

0 2 1 



A 

0\2 

1 

A 

O u 

1 


_A t + A t 

O n 

A‘A X +1 



“2 

1 

ll 


“ 2 “ 

/ 3 + A X A/ = 

1 

2 

1 

,A+A = 

2 


1 

1 

2 


2 


,a;+a; = [2 


A'A +i = [4] 


2 2 ] 
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Hence 


A 2 = 


2 110 0 
12 10 0 
112 0 0 
0 0 0 1 0 
0 0 0 0 1 
2 2 2 0 0 


2 

2 

2 

0 

0 

4 


Toeplitz matrix 


A matrix in which each descending diagonal from left to right is constant is called a 

Toeplitz matrix or diagonal-constant matrix 


Example 9 


a 1 


The matrix A = 


4 

5 

6 


a 

4 

5 


2 

1 

a 

4 


3 

2 

1 


is a Toeplitz matrix. 


a 


Block Toeplitz matrix 

A blocked matrix in which blocks (blocked matrices) are repeated down the diagonals of 
the matrix is called a blocked Toeplitz matrix. 

A block Toeplitz matrix B has the form 


5(1,1) 

5(1,2) 

5(1,3) 

5(1,4) 

5(1,5) 

5(2,1) 

5(1,1) 

5(1,2) 

5(1,3) 

5(1,4) 

5(3,1) 

5(2,1) 

5(1,1) 

5(1,2) 

5(1,3) 

5(4,1) 

5(3,1) 

5(2,1) 

5(1,1) 

5(1,2) 

5(5,1) 

5(4,1) 

5(3,1) 

5(2,1) 

5(1,1) 


Inverses of Partitioned Matrices 

In this section, we will study about the techniques of inverse of blocked matrices. 

Block Diagonal Matrices 

A partitioned matrix A is said to be block diagonal if the matrices on the main diagonal 
are square and all other position matrices are zero, i.e. 
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D, 0 ... 0 

0 d 2 ... 0 

0 0 ... D k 


( 1 ) 


where the matrices Di, D 2 , Z>a are square. It can be shown that the matrix A in (1) is 

invertible if and only if each matrix on the diagonal is invertible, i.e. 

"A 1 0 ... 0 

A -i = 0 A ' - 0 

_ 0 0 ... D k r\ 


Example 10 Let A be a block diagonal matrix 


'1 

2 

0 

0 

0" 

1 

-1 

0 

0 

0 

0 

0 

1 

1 

0 

0 

0 

5 

4 

0 

0 

0 

0 

0 

2 


Find A 1 . 


Solution 


There are three matrices on the main diagonal; two are 2x2 matrices and one 
is 1 x 1 matrix. 

In order to find A -1 , we evaluate the inverses of three matrices lie in main diagonal of A. 

2 ^ . (I A 

1 -1 


Let^ n = 


A -- 
9 yi 22 


V 


V5 4 y 


and A n = (2) are matrices of main diagonal of A. Then 


A -1 _ Ad M 11 
Ai ~' 


A, 


-1 -2 
V-l \j 
-3 


(1 2 ^ 
3 3 

I z ! 
v3 T; 
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There are three matrices on the main diagonal - two 2x2 matrices and one lxl matrix. 



A partitioned square matrix A is said to be block upper triangular if the matrices on the 
main diagonal are square and all matrices below the main diagonal are zero; that is, the 
matrix is partitioned as 
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A = 


^11 ^12 
O A 22 


o o 


A 


ifc 


A, 


2 k 


A, 


kk 


where the matrices A n , A 22 ,..., A kk are 


square. 


Note The definition of block lower triangular matrix is similar. 

Here, we are going to introduce a formula for finding inverse of a block upper triangular 
matrix in the following example. 

Example 12 

Let A be a block upper triangular matrix of the fonn 


A = 


A l j A u 


0 A 


- 22 . 


where the orders of An and A 22 are p x p and qxq respectively. Find A 


Solution 


Let B = 


11 12 1 be inverse of A .i.e. A' 1 = B , then 

f?2i B 22 


AB = 


"4i 

A\i 

~B n 

B\2 


'Ip 

o~ 

o 

A 22 _ 

_B 2 \ 

B 22 


o 

I q _ 


A U B n + A 12 B 21 

A w B \2 + A l2 B 22 


" I P 

°1 

A 22 B 2l 

A Ft 

^ L 22 1J 22 


o 

1 


By comparing corresponding entries, we have 

+ A 12 B 2\~ I p 


A\\B\2 ^\ 2 B 22 o 


( 1 ) 

( 2 ) 
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A 22 B 2l = O (3) 

A 22 B 22 — I q (4) 

Since A 22 is a square matrix, so by Invertible Matrix Theorem, we have A 22 is invertible. 
Thus by eq.(4), B 22 = Now by eq. (3), we have B 2l = A 22 0 = O . From eq.(l) 
A\ 1 | + O = I p 

=> A n B n =I P 

=> 5 n = Ar 1 

Finally, form (2), 

A\\B \ 2 / 1 1 2 B 22 A 12 A 2 2 cind B l2 - 4 11 Aj 2 A 22 

Thus 


A~ l = 

~ 4 i 

A \2 

-1 

r a~ x 

Ai 

An ^12^22 


0 

^22 _ 


0 

1 

V 


Example 13 

Find A' 1 of 


19-50 
3 3 3 -2 

A - 

0 0 7 0 

0 0 3 1 

Solution 

Let partition given matrix A in form 


“1 

9 

-5 

0 " 

3 

3 

3 

-2 

0 

0 

7 

0 

0 

0 

3 

1 


Put A u 


"1 9" 


~-5 0" 


"7 0" 


? -Ai 9 — 


and A„ 0 = 


3 3 

* 12 

1 

<N 

1 

00 

_ 1 

22 

3 1 


Thus A n ' 


1 AdjA 

" 3 -9" 

-3 1 j 


f 1 

-8 

9 > 

24 

andA 22 1 = 

ii ol 

7 

det(A) 

-24 


1 

l 8 

1 

-24 j 

± ! 

L 7 J 
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Moreover, 


A - 1 A A - 

'-'At ^12^22 


f 1 

9 " 



" 1 

0 


r -4 

3 " 

-8 

24 

"-5 

0 " 

7 


~T 

4 

1 

1 

3 

-2 

-3 

1 


1 

-1 

( 8 

-24j 



_ 7 


v 4 

12, 


So by (5), we have 


A- 1 = 


-1 3 

8 8 

I zl 

8 24 

0 0 
0 0 


-4 

~T 

i_ 

4 


3 

4 
-1 
12 


1 

7 

-3 

7 


- 0 


— 1 


Example 14 


Confirm that A = 


4 

3 

0 

0 


7 

5 

0 

0 


-5 

3 

7 

3 


3 

-2 

2 

1 


is an invertible block upper triangular matrix, and 


then find its inverse by using formula (9). 

Solution The matrix is block upper triangular because it can be partitioned into form 



"At 

A 12 

•• Afc 

A = 

0 

A 2 

•• Afc 


0 

0 . 

•• Afc 


as 


where 

Now 


A = 


Ai = 


A A = 


“4 

7 

-5 

3 " 

3 

5 

3 

-2 

0 

0 

7 

2 

0 

0 

3 

1 


A, A, 


ni 

O 


12 


A, 


22 


"4 7] 

, 2^-1 0 — 

"-5 

3 " 

A _ 

"7 2" 

3 5 

> 12 

3 

-2 

? ^ a 22 

3 1 


-5 7 

3 -4 


and A, 


22 


1 -2 
-3 7 
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Moreover, -A ] , 1 A 12 A 22 1 


1 

1 

cn 

-0 

_i 

—i 

l 

cn 

oo 

_i 

" 1 -2" 


-133 

295 " 

-1 

^|- 

1 

oo 

_i 

-1 

or 

i 

co 

_i 

i— 

i 

oo 

-0 

1_ 


78 

-173 


So it follows from (9) that the inverse of A is A 1 


NUMERICAL NOTES 


-5 7 -133 295 

3 -4 78 -173 

0 0 1 -2 
0 0-3 7 


1. When matrices are too large to fit in a computer’s high-speed memory, 
partitioning permits a computer to work with only two or three sub¬ 
matrices at a time. For instance, in recent work on linear programming, a 
research team simplified a problem by partitioning the matrix into 837 
rows and 51 columns. The problem’s solution took about 4 minutes on a 
Cray supercomputer. 

2. Some high-speed computers, particularly those with vector pipeline 
architecture, perform matrix calculations more efficiently when the 
algorithms use partitioned matrices. 

3. The latest professional software for high-performance numerical linear 
algebra, LAPACK, makes intensive use of partitioned matrix calculations. 


The exercises that follow give practice with matrix algebra and illustrate typical 
calculations found in applications. 
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Exercises 


In exercises 1 to 3, the matrices A, B, C, X, Y, Z, and I are all n x n and satisfy the 
indicated equation. 


~ A 

B~ 

7 

O' 


"0 

1 

C 

0 

X 

Y\ 


z 

0 




' A 

z" 



"X 

0 

0~ 

0 

0 

_ 

7 

Y 

0 

I 




0 



- 1 

B 

I 




'X 

0" 

A 

O' 


7 

0" 

Y 

z 

B 

C 


0 

I 


4. Suppose that An is an invertible matrix. Find matrices X and Y such that the product 
below has the form indicated. Also compute B 22 - 


7 

0 

0 " 

"4i 

Ai 1 


~B n 

B\2 

X 

I 

0 

A 21 

A.2 

= 

0 

^22 

Y 

0 

I 

Ai 

Ai\ 


0 

By2 _ 



7 

0 

O' 


7 

0 

0" 

5. The inverse of 

C 

I 

0 

is 

Z 

I 

0 


A 

B 

I 


X 

Y 

I 


Find X, Y and Z. 


6. Find the Inverse of matrix A. 


7. Show that 


I 

A 


"8 

3 

0 

0 

0" 

1 

4 

0 

0 

0 

0 

0 

3 

1 

0 

0 

0 

5 

1 

0 

0 

0 

0 

0 

8_ 


0 

/ 


is invertible and find its inverse. 


8. Compute X T X, when X is partitioned as [Xj X 2 ]. 

In exercises 9 and 10, determine whether block multiplication can be used to compute 
the product using the partitions shown. If so, compute the product by block 
multiplication. 
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9. (a) 


"-1 2 

0 -3 

1 5" 

4 2 

"-2 1 

-3 5 

4 " 

2 

7 -1 

0 3 

5 

-3 

1 5 

6 1 


(b) 


"-1 

2 

1 

5' 

0 

-3 

4 

2 

1 

5 

6 

1 


"-2 

1 

4 ' 

-3 

5 

2 

7 

-1 

5 

0 

3 

-3_ 


10.(a) 


"3 -1 0 

-3" 

"2 -4 ll 

3 0 2 

(b) 

"2 -5" 

1 3 

"2 -1 

3 -4" 

2 1 4 

5 

1 -3 5 

0 

5 

0 1 

5 7 



2 1 4 


1 

4 




11. Compute the product 


"3 

1 ' 

" 1 2 

0 " 

2 

-4 

-1 6 

2 


using the column row-rule, and check 


your answer by calculating the product directly. 

In exercises 11 and 12, find the inverse of the block diagonal matrix A. 


12.(a) A = 


2 10 0 

3 2 0 0 

0 0 3 4 

0 0 1-1 


13. (a) A = 


5 

4 

0 

0 


1 

1 

0 

0 


0 0 
0 0 
2 -3 

-3 5 


(b) 


(b) 


2 

1 

0 

0 

0 

0 

1 

3 

0 

0 


0 

0 

5 

0 

0 

0 

2 

7 

0 

0 


0 0 
0 0 
0 0 
2 7 
1 4 
0 0 
0 0 
0 0 
4 9 
1 2 


14. Find the inverse of the block upper triangular matrix A. 


"2 

1 

3 

-6~ 


'-1 

-1 

2 

5“ 

1 

1 

7 

4 

(ii) A = 

2 

1 

-3 

8 

0 

0 

3 

5 

0 

0 

4 

1 

0 

0 

2 

3 _ 


0 

0 

7 

2 


15. Find Bi, given that 
and 



1- 

1_ 

b 2 1 


l_ 

_i 

1 

o 

1 

CT 

1- 

O 

i_ 


I 

o 

i 

cf 
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A x = ,B 2 = ,Cj = 

1 [o lj 2 [i 2 J L 1 - 1 

[2 0 ] [ 10 ] [2 1 

A,= ,C 2 = ,B,= 

- 0 2 2 0 2 3 1 3 


16. Consider the partitioned linear system 


“5 

2 

2 

3" 

V 

2 

1 

-3 

1 

x 2 

1 

0 

4 

1 

x 3 

0 

1 

0 

2 

_ X 4_ 


Solve this system by first expressing it as 


A B u 
I D v 


b . Au + Bv = b 

or equivalently, 

0 u + Dv = 0 


next solving the second equation for u in terms of v, and then substituting in the first 

equation. Check your answer by solving the system directly. 

r \a b 

-3 1 2 

17. Let A = and B= c d . 

1 -4 5 

l e /_ 

Verify that AB = col 1 (A)row 1 (B) + col 2 {A)row 2 (B) + col 3 (A)row 3 (B) 

[4 7 -5 3] 


18. Find inverse of the matrix A = 


3 5 3 -2 

0 0 7 2 

0 0 3 1 
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Lecture 15 

Matrix Factorizations 


Matrix Factorization 

A factorization of a matrix as a product of two or more matrices is called Matrix 
Factorization. 

Uses of Matrix Factorization 

Matrix factorizations will appear at a number of key points throughout the course. This 
lecture focuses on a factorization that lies at the heart of several important computer 
programs widely used in applications. 

LUFactorization or LU-decomposition 

LU factorization is a matrix decomposition which writes a matrix as the product of a 
lower triangular matrix and an upper triangular matrix. This decomposition is used to 
solve systems of linear equations or calculate the determinant. 

Assume A is an mxn matrix that can be row reduced to echelon form, without row 
interchanges. Then A can be written in the form A = LU, where L is an mxm lower 
triangular matrix with l’s on the diagonal and U is an mxn echelon form of A. For 
instance, such a factorization is called LU factorization of A. The matrix L is invertible 
and is called a unit lower triangular matrix. 


"1 

0 

0 

0 " 

• 

* 

* 

* 

*“ 

* 

1 

0 

0 

0 

• 

* 

* 

* 

* 

* 

1 

0 

0 

0 

0 

• 

* 

* 

* 

* 

1 

0 

0 

0 

0 

0 


L U 

LU factorization. 


Remarks 

1) If A is the square matrix of order m, then the order of both L and U will also be m. 

2) In general, not every square matrix A has an LU-decomposition, nor is an LU- 
decomposition unique if it exists. 

Theorem If a square matrix A can be reduced to row echelon form with no row 
interchanges, then ,4 has an LU-decomposition. 

Note 

The computational efficiency of the LU factorization depends on knowing L and U. The 
next algorithm shows that the row reduction of A to an echelon form U amounts to an LU 
factorization because it produces L with essentially no extra work. 

An LU Factorization Algorithm 

Suppose A can be reduced to an echelon form U without row interchanges. Then, since 
row scaling is not essential, A can be reduced to U with only row replacements, adding a 
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multiple of one row to another row below it. In this case, there exist lower triangular 
elementary matrices E/, E p such that 



E p ... EiA = U 

(1) 

So 

A = (E p ...E 1 ) 1 U = LU 


Where 

L = (E p ... E,)-' 

(2) 


It can be shown that products and inverses of unit lower triangular matrices are also unit- 
lower triangular. Thus, L is unit-lower triangular. 

Note that the row operations in (1), which reduce A to U, also reduce the L in (2) to /, 
because E p ... E/L = (Ep...E t )(E n ... Ej)' 1 = I. This observation is the key to 
constructing L. 


Procedure for finding an L [/-decomposition 

Step 1: Reduce matrix .4 to row echelon form U without using row interchanges, keeping 
track of the multipliers used to introduce the leading l’s and the multipliers used to 
introduce zeros below the leading l’s. 

Step 2: In each position along the main diagonal of L, place the reciprocal of the 
multiplier that introduced the leading 1 in that position in U. 

Step 3: In each position below the main diagonal of L, place the negative of the 
multiplier used to introduce the zero in that position in U. 


Step 4: Form the decomposition^ = LU. 


Example 1 


Find an L [/-decomposition of 


A = 


6-2 0 
9-11 
3 7 5 


Solution We will reduce A to a row echelon fonn U and at each step we will fill in an 
entry of L in accordance with the four-step procedure above. 



~6 -2 0 1 


“* 0 0" 

A = 

9 -1 1 


* * o 


3 7 5 


* * * 


'0 

9 

3 

1 

i 

0 


]_ 

3 


-1 

7 


i 

3 


2 

8 


0 

1 

5 

0 

1 

5 


<— multiplier = j- 


<— multiplier = -9 
<— multiplier = -3 


* denotes an unknown entry of L. 
"6 0 0 " 

* * 0 
* H= 

"6 0 0 " 

9 * 0 

3 * * 
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1 -1 o' 

0 1 i 

9 

<— multiplier = \ 

"6 0 0" 

9 2 0 

Z, 

0 8 5 


3 * * 

1 -} o" 


"6 0 0" 

0 1 i 

<— multiplier = -8 

9 2 0 

-1 

°l 

o 


3 8 * 



1 “I °" 


o 

o 

SO 

1_ 

U = 

0 1 i 

<— multiplier = 1 L = 

9 2 0 


1 

o 

o 

_1 


3 8 1 


(No actual operation is performed here since there is already a leading 1 in the third row.) 



"6 

0 

0" 

“l 

1 

3 

0" 

A = LU = 

9 

2 

0 

0 

1 

1 

2 


3 

8 

1 

0 

0 

1 


OR 


Solution We will reduce A to a row echelon fonn U and at each step we will fill in an 
entry of L in accordance with the four-step procedure above. 



1 

l_ 


"* 0 0" 

A = 

9-11 


* * Q 


3 7 5 


* * * 


6 _ 

~6 

9 

3 


-_ 2 _ 0 _ 
~6 6 
-1 1 

7 5 



* denotes an unknown entry of L. 

"6 0 0 " 

* * 0 


1 

9 

3 


3 

-1 1 
7 5 
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3 

9-9(1) 

3-3(1) 7-3(j ) 


0 

1-9(0) 

5-3(0) 


R 2 -9R i 
R 3 -3R l 


6 0 0 
9 * 0 

3 * * 
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"l -i o" 


o 

o 

SO 

1_ 

u = 

0 1 i 

<— multiplier = 1 L = 

9 2 0 


1 

o 

o 

S 

1_ 


3 8 1 



"6 

0 

0" 

"l 

1 

3 

o' 

A = LU = 

9 

2 

0 

0 

1 

1 

2 


3 

8 

1 

0 

0 

1 


Example 2 Find an LU factorization of A = 


2 

6 

2 

4 

-6 


-4 

-9 

-7 

-2 

3 


-2 3 

-5 8 

-3 9 

-2 -1 
3 4 


Solution 


zf = 


2 

6 

2 

4 

-6 


-4 -2 3 

-9 -5 8 

-7 -3 9 

-2 -2 -1 
3 3 4 


1 -2 

6 -9 

2 -7 

4 -2 

-6 3 


-1 

-5 

-3 

-2 

3 


3 

2 

8 

9 


<—multiplier — 


-1 

4 


1 -2 

0 3 
0 -3 
0 6 
0 -9 


-1 

1 

-1 

2 

-3 


3 

2 

-1 

6 

-7 

13 


<— multiplier - 6 
<— multiplier - 2 
<— multiplier - 4 
<— multiplier 6 


1 0 0 0 0 

* 1 0 0 0 

* * 1 0 0 

* * * ^ 0 

* 1 

* denotes an unknown entry of L. 

"2 0 0 0 0 " 

* 1 0 0 0 

* * 1 0 0 

* * * | q 

* 1 

"2 0 0 0 O' 

6 10 0 0 
2*100 
4**10 
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1 -2 -1 


3 

0 -3 -1 
0 6 2 
0 -9 -3 

1 -2 -1 

0 1 I 

3 

0 0 0 
0 0 0 
0 0 0 

1 -2 -1 

0 1 I 

3 

0 0 0 
0 0 0 
0 0 0 

1 -2 -1 


0 0 0 
0 0 0 
0 0 0 



<— multiplier — 


<— multiplier 3 
^multiplier -6 
<— multiplier 9 


<—multiplier — 


<—multiplier-\() 


2 0 0 0 0 

6 3 0 0 0 

2*100 
4**10 

^ ^ J 

2 0 0 0 0 

6 3 0 0 0 

2-3100 
4 6*10 

-6 -9 * * 1 


2 0 0 0 0 

6 3 0 0 0 

2-3100 
4 6 0 -5 0 

-6-9 0* 1 


L = 


2 0 
6 3 

2 -3 

4 6 

-6 -9 


0 0 0 

0 0 0 

1 0 0 

0-5 0 

0 10 1 


Thus, we have constructed the L [/-decomposition 


A = LU = 


2 0 0 0 

6 3 0 0 

2-310 
4 6 0 -5 

-6 -9 0 10 


0 1 

0 0 
0 0 
0 0 


-1 

\_ 

3 

0 

0 

0 


3 

2 

1 

'3 

5 

1 

0 
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Example 3 Find 7 [/-decomposition of A = 


6 

3 


-2 -4 

-3 -6 

-12 8 21 

-6 0 -10 


4 

1 

-8 

7 


Solution 



6 

-2 

-4 

4 ' 


'1 

0 

0 

0" 


3 

-3 

-6 

1 


* 

1 

0 

0 

A = 

-12 

8 

21 

-8 

L = 

* 

* 

1 

0 


-6 

0 

-10 

7 


* 

* 

* 

1 


* denotes an unknown entry of L. 


! _l _2 2] 

<— multiplier — 

"6 0 0 0" 

3 3 3 

3-3-6 1 

-12 8 21 -8 

-6 0 -10 7 

6 

* 1 0 0 

* * 1 0 

* * * 2 


1 

0 

0 

0 


1 2 2” 



Q Q Q 


" 6 0 0 0" 

O O O 

-2 -4 -1 

<— multiplier - 3 

3 10 0 

4 13 0 

<— multiplier 12 

-12 * 1 0 

-2 -14 11 

<— multiplier 6 

-6**1 


1 

0 

0 

0 


3 
1 

4 

-2 


_2 

3 

2 

13 

-14 


2 

3 

1 

2 

0 

11 


<— multiplier 


1 

2 


6 0 0 0 
3 -2 0 0 

-12 * 10 
- 6**1 
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1 

0 

0 

0 

1 

0 

0 

0 


1 _2 
3 3 

1 2 

0 5 

0 -10 
1 _2 
3 3 

1 2 

0 1 
0 -10 


2 

3 

1 

2 

-2 

12 

2 

3 

1 

2 

_2 

5 

12 


<— multiplier - 4 
<— multiplier 2 


<— multiplier — 


6 0 0 0 
3 -2 0 0 

-12 4 1 0 

- 6 - 2*1 


6 0 0 0 
3 -2 0 0 

-12 4 5 0 

- 6 - 2*1 


1 

0 

0 

0 


L l 

1 "7 

1 2 

0 1 
0 0 


2 

J 

1 

2 
2 
5 
8 


<- multiplier 10 


6 0 0 0 
3-200 
-12 4 50 

-6 -2 -10 1 


1 


U = 


0 


1 

3 

1 


0 0 


0 0 


2 

3 

2 

1 

0 


2 

3 

1 

2 

2 

"5 

1 


<— multiplier — 


6 0 
3 -2 

-12 4 

-6 -2 


0 0 
0 0 
5 0 

-10 8 


Thus A = LU = 


6 

3 

-12 

-6 


0 

-2 

4 

-2 


0 

0 

5 

-10 


0 

0 

0 

8 


1 

0 

0 


1 

3 

1 

0 


0 0 
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Example 4 Find an LU factorization of A = 


2 

-4 

2 

-6 


4 

-5 

-5 

0 


-1 5 

3 -8 

-4 1 

7 -3 


-2 

1 

8 

1 


Solution 



" 2 

4 

-1 

5 

-2 


'1 

0 

0 

0" 


-4 

-5 

3 

-8 

1 


* 

1 

0 

0 

A = 

2 

-5 

-4 

1 

8 

L = 

* 

* 

1 

0 


-6 

0 

7 

-3 

1 


* 

* 

* 

1 


2 

-4 -5 3 

2 -5 -4 

-6 0 7 


5 

2 

-8 

1 

-3 


* denotes an unknown entry of L. 


<— multiplier — 

2 

"2 

0 

0 

0" 


* 

1 

0 

0 


* 

* 

1 

0 


* 

* 

* 

1 


1 2 

0 3 
0 -9 
0 12 


_ 1 
~2 

1 

-3 

4 


5 

2 

2 

-4 

12 


-1 

-3 

10 

-5 


<— multiplier 4 
<— multiplier - 2 
<— multiplier 6 


2 0 0 0 
-4100 
2*10 
- 6**1 


1 2 

0 1 

0 -9 
0 12 


2 

1 

3 

-3 

4 


5 

2 

2 

3 

-4 

12 


-1 

1 <— multiplier ^ 

10 

-5 


2 0 0 0 
-4 3 0 0 
2*10 
- 6**1 


1 2 

0 1 

0 0 
0 0 


1 

~2 

3 

0 

0 


5 

2 

2 

3 
2 

4 


-1 

-1 

1 <— multiplier 9 

7 J <— multiplier -12 


2 0 0 0 
-4 3 0 0 

2-910 
-6 12 * 1 
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u = 


12-15 _! 
2 2 


0 1 
0 0 
0 0 


1 

3 

0 

0 


5 -i 

3 

2 1 

i l 

4 


<— multiplier 


2 0 0 
-4 3 0 

2 -9 1 

-6 12 0 


0 

0 

0 

4 


Thus 


A = LU = 


2 

-4 

2 

-6 


0 

3 

-9 

12 


0 

0 

1 

0 


0 

0 

0 

4 


1 

0 

0 


2 

1 

0 


0 0 


1 £ 

2 2 

I £ 

3 3 
0 2 

0 1 


-1 

-1 

1 

7 

4 


Matrix Inversion by Z, [/-Decomposition 

Many of the best algorithms for inverting matrices use /.[/-decomposition. To understand 
how this can be done, let .4 be an invertible nxn matrix, let A = [x, x 2 ••• x n ] be 

its unknown inverse partitioned into column vectors, and let I = \e l e 2 e n 1 be then 

nxn identity matrix partitioned into column vectors. The matrix equation AA~ l =/ can 
be expressed as 

A i x i *2 ••• *,J = h ••• e„] 

[A A ••• A,] = h e 2 ••• e „] 

which tells us that the unknown column vectors of A can be obtained by solving the n- 
linear systems. 

A =e i, Ax 2 =e 2 ,---, Ax n =e n (1*) 

As discussed above, this can be done by finding an L [/-decomposition of A, and then 
using that decomposition to solve each of the n systems in (1*). 


Solving Linear System by LU-Factorization 

When A =LU, the equation Ax = b can be written as L (Ux) = b. Writings for Ux, we can 
find x by solving the pair of equations; Ly = b and Ux=y (2*) 

First solve Ly = b for y and then solve Ux = y for x. Each equation is easy to solve 
because L and U are triangular. 
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Procedure 

Step 1: Rewrite the system A x = b as LUx = b (3 *) 

Step 2: Define a new unknown y by letting Ux =y (4*) 

And rewrite (3*) as L y = b 

Step 3: Solve the system Ly = b for the unknown y. 

Step 4: Substitute the known vector y into (4*) and solve for x. 

This procedure is called the method of L [/-Decomposition. 

Although L [/-Decomposition converts the problem of solving the single system A x = b 
into the problem of solving the two systems, L y = b and U x = y, these systems are easy 
to solve because their co-efficient matrices are triangular. 


Example 5 Solve the given system (Ax =b) by ££/-Decomposition 
2x l + 6x 2 + 2x 3 = 2 

-3xj-8x 2 =2 (1) 

4xj + 9x, + 2x 3 = 3 


Solution We express the system (1) in matrix form: 


1- 

SO 

<N 

l_ 

x x 


“2“ 

-3 -8 0 

x 2 

= 

2 

4 9 2 

x 3 


3 

A 

X 

= 

b 


We derive an L [/-decomposition of A 



' 2 

6 

2] 



"1 0 

c 

A = 

-3 

-8 

0 

L = 

* 1 

c 


4 

9 

2 



* * 

1 

" 1 

3 

f 


-t 

"2 

0 O' 


-3 

-8 

0 

<— multiplier — 

* 

1 0 


4 

9 

2 



* 

* 1 


"1 

3 

1 ' 



" 2 

0 0" 

0 

1 

3 

<— multiplier 3 

-3 

1 0 

0 

-3 - 

-2 

<— multiplier - 4 

4 

* 1 

"1 

3 ll 




' 2 

0 

0" 

0 

1 3 

<— multiplier 3 

-3 

1 

0 

0 

0 1 




4 

-3 

1 
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'1 

3 

ll 

<— multiplier — 

7 


' 2 

0 

0 " 

u = 

0 

1 

3 

L = 

-3 

1 

0 


0 

0 

1 



4 

-3 7 


" 2 

6 

2] 


' 2 

0 

0“ 

"1 

3 

f 

-3 

-8 

0 

= 

-3 

1 

0 

0 

1 

3 

4 

9 

2 


4 

-3 

7 

0 

0 

1 


A = L U 

From (2) we can rewrite this system as 


" 2 

0 

0 " 

"1 

3 

f 



' 2 ' 

-3 

1 

0 

0 

1 

3 

x 2 

= 

2 

4 

-3 

7 

0 

0 

1 

x 3 


3 


L U x = b 

As specified in Step 2 above, let us define yi, y 2 andio by the equation 


"1 

3 

1 " 

x i 


Ti 

0 

1 

3 

x 2 

= 


0 

0 

1 

x 3 


_y 3 _ 


U x = y 
which allows us to rewrite (3) as 


i - 

O 

O 

<N 

1_ 

Ti 


'2' 

-3 1 0 


= 

2 

4-3 7 

_t 3 J 


3 

L 

y 

= 

b 


2fi = 2 

or equivalently, as -3 y l +y 2 =2 

4y t ~3y 2 +7y 3 =3 


( 2 ) 


( 3 ) 


( 4 ) 


( 5 ) 


This system can be solved by a procedure that is similar to back substitution, except that 
we solve the equations from the top down instead of from the bottom up. This procedure, 
called forward substitution, yields 

yi = 1, V 2 = 5, y 3 = 2. 

As indicated in Step 4 above, we substitute these values into (4), which yields the linear 
system 


"1 

3 

f 



V 

0 

1 

3 

x 2 

= 

5 

0 

0 

1 

x 3 


2 
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Xj + 3x 2 + x 3 = 1 
or equivalently, x 2 + 3x 3 = 5 

x 3 = 2 


Solving this system by back substitution yields x; = 2, X 2 = -1, x^ = 2 
Example 6 It can be verified that 

A = 


" 3 

-7 

-2 

2 ' 


' 1 

0 

0 

0 " 

"3 

-7 

-2 

2~ 

-3 

5 

1 

0 


-1 

1 

0 

0 

0 

-2 

-1 

2 

6 

-4 

0 

-5 


2 

-5 

1 

0 

0 

0 

-1 

1 

-9 

5 

-5 

12 


-3 

8 

3 

1 

0 

0 

0 

-1 


Use this LU factorization of A to solve Ax = b, where b = 


-9 

5 

7 

11 


= LU 


Solution The solution of Ly = b needs only 6 multiplications and 6 additions, because the 
arithmetic takes place only in column 5. (The zeros below each pivot in L are created 
automatically by our choice of row operations.) 


[L 


" 1 

0 

0 

0 

-9 


'1 

0 

0 

0 

-9 

-1 

1 

0 

0 

5 


0 

1 

0 

0 

-4 

2 

-5 

1 

0 

7 


0 

0 

1 

0 

5 

-3 

8 

3 

1 

11 


0 

0 

0 

1 

1 


v] 


Then, for Ux = y, the “backwards” phase of row reduction requires 4 divisions, 6 
multiplications, and 6 additions. (For instance, creating the zeros in column 4 of [U y] 
requires 1 division in row 4 and 3 multiplication - addition pairs to add multiples of row 
4 to the rows above.) 


"3 

-7 

-2 

2 

-9 


'1 

0 

0 

0 

3 ' 


" 3 " 

0 

-2 

-1 

2 

-4 


0 

1 

0 

0 

4 

, x = 

4 

0 

0 

-1 

1 

5 


0 

0 

1 

0 

-6 


-6 

0 

0 

0 

-1 

1 


0 

0 

0 

1 

-1 


-1 


To find x requires 28 arithmetic operations, or “flops” (floating point operations), 
excluding the cost of finding L and V. In contrast, row reduction of [A b ] to [I x] takes 
62 operations. 

Numerical Notes 

The following operation counts apply to an nxn dense matrix A (with most entries 
nonzero) for n moderately large, say, n > 30. 
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3 

1. Computing an LU factorization of A takes about 2n /3 flops (about the same as 
row reducing \A b ]), whereas finding^' 7 requires about 2n flops. 

2. Solving Ly = b and Ux = y requires about 2 n flops, because n x n triangular 

2 

system can be solved in about n flops. 

3. Multiplication of b by A' also requires about 2 n flops, but the result may not be 
as accurate as that obtained from L and U (because of round off error when 
computing both A' 1 and A~'b). 

4. If A is sparse (with mostly zero entries), then L and U may be sparse, too, 
whereas A' 1 is likely to be dense. In this case, a solution of Ax = b with an LU 
factorization is much faster than using A' 1 . 

Example 7(Gaussian Elimination Performed as an LU-Decomposition) 

In Example 5, we showed how to solve the linear system 

2 6 2 x 1 2 

—3 —8 0 x 2 = 2 (6) 

4 9 2_||_x 3 J [3 

using an L [/-decomposition of the coefficient matrix, but we did not discuss how the 
factorization was derived. In the course of solving the system, we obtained the 

"f 

intermediate vector y = 5 by using forward substitution to solve system (5). 

2 

We will now use the procedure discussed above to find both the L [/-decomposition and 
the vector y by row operations on the augmented matrix for (6). 

" 2 6 2 2l [* 0 O' 

[^4|h]= -3 -8 0 2 * * 0 =L (* = unknown entries ) 

4 9 2 3 * * * 

13 11] [2 0 0 

-3 -8 0 2 * * 0 

4 9 2 3 * * * 

13 1 1 ] [200 

0 13 3 -3*0 

0 -3 -2 -lj ^4 * * 

"13 11 ] [ 2 0 0 " 

0 13 5 -3 10 

0 0 7 14j [4 -3 * 
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"1 3 1 

r 


"2 00 " 

0 1 3 

5 


-3 1 0 

1 

0 

0 

2 


4-37 


These results agree with those in Example 5, so we have found an Z, [/-decomposition of 
the coefficient matrix and simultaneously have completed the forward substitution 
required to find y. 

All that remains to solve the given system is to solve the system Ux = y by back 
substitution. The computations were performed in Example5. 

A Matrix Factorization in Electrical Engineering 

Matrix factorization is intimately related to the problem of constructing an electrical 
network with specified properties. The following discussion gives just a glimpse of the 
connection between factorization and circuit design. 


Suppose the box in below Figure represents some sort of electric circuit, with an input 


and output. Record the input voltage and current by 


(with voltage v in volts and 


current i in amps), and record the output voltage and current by 


. Frequently, the 


transfonnation 

matrix, such that 


is linear. That is, there is a matrix A, called the transfer 


"v 2 l 




= A 


UJ 


L z i J 


i 

-ki 

1 | 


input v | 
terminals 


Figure 



v 2 output 


terminals 


A circuit with input and output terminals. 


Above Figure shows a ladder network, where two circuits (there could be more) are 
connected in series, so that the output of one circuit becomes the input of the next circuit. 
The left circuit in Figure is called a series circuit, with resistance R / (in ohms); 


The right circuit is a shunt circuit, with resistance R 2 . Using Ohm’s law and Kirchhoff s 
laws, one can show that the transfer matrices of the series and shunt circuits, respectively, 
are: 
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1 -R, 
0 1 


and 


Transfer matrix 
of series circuit 


1 0 
-1/R 2 1 

Transfer matrix 
of shunt circuit 


Example 8 

a) Compute the transfer matrix of the ladder network in the above Figure . 


b) Design a ladder network whose transfer matrix is 


1 

-0-5 


-8 

5 


Solution 

a) Let A / and A 2 be the transfer matrices of the series of the series and shunt circuits, 
respectively. Then an input vector x is transformed first into Ajx and then into 
A 2 (Aix). The series connection of the circuits corresponds to composition of linear 
transfonnations; and the transfer matrix of the ladder network is (note the order) 


A 2 A\ 


1 0 " 

"1 



1 


-1 ir 2 1 

0 

1 


-1 !R 2 

1 + R l /R 2 


b) We seek to factor the matrix 


1 -8 
-0-5 5 


( 6 ) 


into the product of transfer matrices, such 


as in ( 6 ). So we look for R 7 and R 2 to satisfy 


1 

-*1 


1 

-8" 

-1/R 2 

1 + R l /R 1 


-0-5 

5 _ 


From the (1,2)- entries, Ri = 8 ohms, and from the (2, 1) - entries, MR 2 = 0.5 ohm and 
R 2 = 1/0.5 = 2 ohms. With these values, the network has the desired transfer matrix. 

Note 

A network transfer matrix summarizes the input-output behavior (“Design 
specifications”) of the network without reference to the interior circuits. To physically 
build a network with specified properties, an engineer first determines if such a network 
can be constructed (or realized). Then the engineer tries to factor the transfer matrix into 
matrices corresponding to smaller circuits that perhaps are already manufactured and 
ready for assembly. In the common case of alternating current, the entries in the transfer 
matrix are usually rational complex-valued functions. A standard problem is to find a 
minimal realization that uses the smallest number of electrical components. 
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Exercises 

Find an LU factorization of the matrices in exercises 1 to 8. 





' 3 

-1 

2~ 


"2 5 1 





1 . 

-3 -4 

2. 

-3 

-2 

10 


u ~ 


9 

-5 

6 


1 3-5-3 

-1-58 4 

4 2-5-7 

-2-47 5 

2 -6 6 

-4 5 -7 

3 5-1 

-6 4 -8 

8-3 9 







' 2 

-4 

-2 

3 ' 

' 1 

4 

-1 

5 “ 


6 

-9 

-5 

8 

3 

7 

-2 

9 

8. 

2 

-7 

-3 

9 

-2 

-3 

1 

-4 


4 

-2 

-2 

-1 

-1 

6 

-1 

7 _ 











-6 

3 

3 

4 


3-6 3 
6-7 2 
-17 0 


5. 


2 -4 4-2 

6 -9 7-3 

-1-4 8 0 


Solve the equation = b by using L [/-factorization. 


9. A = 

"3 -7 -2l 

-3 5 1 

,b = 

“-7“ 

5 

10. A = 

"4 3 -5l 

-4 -5 7 

,b = 

' 2 ' 

-4 


6-4 0 


2 


8 6-8 


6 




' 2 

-1 

2 ' 


'll 



'2 

-2 

4] 


' 0 " 

11. 

zf = 

-6 

0 

-2 

,b = 

0 

12. 

zf = 

1 

-3 

1 

,b = 

-5 



8 

-1 

5 


4 



3 

7 

5 


7 




' 1 

-2 

-4 

-3] 


'll 


' 1 

3 

4 

0 ' 


' 1 ' 



2 

-7 

-7 

-6 


7 


-3 

-6 

-7 

2 


-2 

13. 

zl = 

-1 

2 

6 

4 

,b = 

0 

14. zl = 

3 

3 

0 

-4 

,b = 

-1 



-4 

-1 

9 

8 


3 


-5 

-3 

2 

9 


2 
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Lecture 16 


Iterative Solutions of Linear Systems 

Consistent linear systems are solved in one of two ways by direct calculation (matrix 
factorization) or by an iterative procedure that generates a sequence of vectors that 
approach the exact solution. When the coefficient matrix is large and sparse (with a high 
proportion of zero entries), iterative algorithms can be more rapid than direct methods 
and can require less computer memory. Also, an iterative process may be stopped as soon 
as an approximate solution is sufficiently accurate for practical work. 

General Framework for an Iterative Solution of Ax = b 

Throughout the section, A is an invertible matrix. The goal of an iterative algorithm is 
to produce a sequence of vectors, 

v (0) (1) (k) 

A/ ^ A ^ ^ A- 9*** 

that converges to the unique solution say x of Ax = b, in the sense that the entries 
... * 
in x ’ are as close as desired to the corresponding entries in x for all k sufficiently 

large. 

To describe a recursion algorithm that produces x a+1> from x CA) , we write A = M -N 
for suitable matrices M and N, and then we rewrite the equation Ax = b as Mx - Nx = b 
and 

Mx = Nx + b 


If a sequence { x (A ° } satisfies 

Mx (k+l) = Nx (k) +b (k = 0,1,...) 


( 1 ) 


and if the sequence converges to some vector x , then it can be shown that Ax = b. [The 

* 

vector on the left in (1) approaches Mx , while the vector on the right in (1) approaches 

H* ... ^ ^ ^ 

Nx +b. This implies that Mx = Nx +b and Ax = b. 


For x (A+1) to be uniquely specified in (1), M must be invertible. Also, M should be 
chosen so that x (A+1) is easy to calculate. There are two iterative methods below to 
illustrate two simple choices for M. 


1) Jacobi’s Method 

This method assumes that the diagonal entries of A are all nonzero. 

Choosing Mas the diagonal matrix formed from the diagonal entries of A. So next 
N = M-A, 

(1) => Mx (k+l) = (M - A)x <k) +b (k = 0,1,...) 


For simplicity, we take the zero vector as x <0) as the initial approximation. 
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Example 1 

Apply Jacobi’s method to the system 


10 xi + X2 - X3 = 18 
xi + 15x2 + x? = -12 

-xi + X2 + 20x3 = 17 ( 2 ) 

Take x (0) = (0, 0, 0) as an initial approximation to the solution, and use six iterations (that 
is, compute x (1) , ... , x (6) ). 


Solution 



x , 


Vl 

For some k , let x <k) = 

x 2 

_x 3 _ 

=(xj, X2, X3) and x (k+1) = 

V 2 

_t 3 _ 


Firstly we will construct M and N from A. 
Here 


=(yi> V2, *) 


A = 


10 

1 


-1 


1 

15 

1 


-1 

1 

20 


Its diagonal entries will give 



"10 

0 

0 ' 


M = 

0 

15 

0 

and 


0 

0 

20 





"10 

1 

0 

0 


N = 

M- 

■A = 

0 

15 0 

- 




0 

0 20 



Now the recursion: Mx (k+l) - (M - 
implies 


1 

-f 


' 0 

-1 

1 ' 

15 

1 

= 

-1 

0 

-1 

1 

20 


1 

-1 

0 


{k) +b {here k = 0,1,...6) 


"10 

0 

0 ' 

y, 


' 0 

-1 

1 " 



' 18 ' 

0 

15 

0 

y 2 

= 

-1 

0 

-1 

x 2 

+ 

-12 

0 

0 

20 

_t 3 _ 


1 

-1 

0 

x 3 


17 



10 * 


O.Xj - lx 2 + 1 x 3 


' 18 ' 

=> 

15* 

= 

-lx, +0x 2 -x 3 

+ 

-12 


.20* J 


lxj - lx 2 + 0x 3 


17 



" 10 *" 


Oxj - lx 2 + lx 3 

+ 18" 

=> 

15* 

= 

—lXj + 0x o -x 3 

-12 


_ 20 *. 


lxj - lx, + 0x 3 

+ 17 


Comparing the corresponding entries on both sides, we have 
10 yi = -X 2 + X 3 + 18 
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15y2 = -xi -X3—I2 
20y3 = X 1 —X 2 +17 

And 


Vi = (-X 2 + X 3 + 18)/10 

y 2 = (-x 1 -x 3 -12)715 (3) 

V 3 = (X 1 — X 2 + 17)720 


1st Iteration 

For k = 0, put x (0) = (xi, X2, X3) = (0, 0, 0) in (3) and compute 

x (1) = ( y h y 2 , y 3 ) = (18/10, - 12/15, 17/20) = (1.8, -0.8, 0.85) 

2 nd Iteration 


For k= 1, put x (1) =(1.8, -.8, .85) 

yi = [-{ -0.8) + (0.85) + 18]/10 = 1.965 
y 2 = [-(1.8)- (0.85)- 12J/15 = -0.9767 
y s = [(1.8) - (-0.8) + 17]/20 = 0.98 


Thus, x (2) = (1.965, -.9767, .98). 

The entries in x {2) are used on the right in (3) to compute the entries in x <3) , and so on. 
Here are the results, with calculations using MATLAB and results reported to four 
decimal places: 


x (0) 

x (1) 

x (2) 

x (3) 

x (4) 

x (5) 

x (6) 

“o' 


“1.87 


' 1.965 ' 


'1.9957' 


'1.9993' 


'1.9999' 


' 2.0000 ' 

0 


-.8 


-.9767 


-.9963 


-.9995 


-.9999 


-1.0000 

0 


.85 


.98 


.9971 


.9996 


.9999 


1.0000 


If we decide to stop when the entries in x (k) and x <k 11 differ by less than .001, then we 
need five iterations (k = 5). 

Alternative Approach 

If we express the above system as 

,. , „ 18 - x, + x, 

10 xi + x 2 - X 3 = 7S=>Xj =-^ 1 

,, ,, -12-x. -x, 

Xi + 15x2 + X 3 =-72=>x, =- i -- 

15 

, 17 + x, -x, 

-X; + Xi + 20x3 = /7=^>X, =---- 

. 3 20 

the equivalent system is 
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x 2 = 


x 3 = 


18 - x 2 + x 3 

To 

-12 -Xj -x 3 

15 

17 + Xj -x 2 

20 


Now put (x7,X2,Xi)=(0,0,0)= x l0) in the RHS to have 

*i=(18-0+0)/10= 1.80 

X 2 = (-12-0-0)/15 = -0.80 
X 3 = (17+0-0)/20 = 0.85 

Which gives x (1) = (1.80,-0.80,0.85) —put this again on RHS of the equivalent system 
to get 

X/=(18+0.80+0.85)/10 =1.965 
x 2 = (-12-1.80-0.85)/15 = -0.9767 
Xf= (17+1.80+0.80)/20 = 0.98 

So in the similar fashion, we can get the next approximate solutions: x ( 3 ) ,x (4) ,x (5) and x (6) 
Next example will be solved by following this approach. 

Example 2 

Use Jacobi iteration to approximate the solution of the system 

20 Xj + x 2 -x 3 =17 

Xj - 10x 2 + x 3 =13 
-Xj + x 2 + 10 x 3 =18 

Stop the process when the entries in two successive iterations are the same when rounded 
to four decimal places. 


Solution 

As required for Jacobi iteration, we begin by solving the first equation for xi, the second 
for X 2 , and the third for X 3 . This yields 
17 1 1 

x, =-x, -l-x, = 0.85-0.05x, + 0.05x, 

1 20 20 2 20 3 2 3 


13 1 1 , , n , n , 

x, =-1-x, -i-x, =-1.3 + 0. lx. + 0.1x, 

2 10 10 1 10 3 1 3 
18 1 1 

x, =- 1 -x.-x, = 1.8 + 0. lx. -O.lx, 

3 10 10 1 10 2 1 


which we can write in matrix form as 


Xi 


" 0 

-0.05 

0.05' 

Xi 


"0.85' 

x 2 

= 

0.1 

0 

0.1 

x 2 

+ 

-1.3 

*3. 


0.1 

- 0.1 

0 

_*3_ 


. L8 


( 4 ) 


( 5 ) 
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Since we have no special infonnation about the solution, we will take the initial 
approximation to beXj = x 2 = x 3 = 0 . To obtain the first iterate, we substitute these values 
into the right side of (5). This yields 


y, = 


0.85 

-1.3 

1.8 


To obtain the second iterate, we substitute the entries of v; into the right side of (5). This 
yields 



*1 


~ 0 

-0.05 

0.05' 

"0.85" 


"0.85" 


' 1.005 ' 

T2 = 

x 2 

= 

0.1 

0 

0.1 

-1.3 

+ 

-1.3 

= 

-1.035 


x 3 


0.1 

-0.1 

0 

1.8 


1.8 


2.015 


Repeating this process until two successive iterations match to four decimal places yields 


the results in the fol 

[lowing tal 

ble: 


yo 

yi 

yi 

ys 

y 4 

ys 

ye 

y 7 

Xl 

0 

0.8500 

1.0050 

1.0025 

1.0001 

1.0000 

1.0000 

1.0000 

X2 

0 

-1.3000 

-1.0350 

-0.9980 

-0.9994 

-1.0000 

-1.0000 

-1.0000 

X 3 

0 

1.8000 

2.0150 

2.0040 

2.0000 

1.9999 

2.0000 

2.0000 


The Gauss-Seidel Method 

This method uses the recursion (1) with M the lower triangular part of A. That is, M has 
the same entries as A on the diagonal and below, and M has zeros above the diagonal. See 
Fig. 1. As in Jacobi’s method, the diagonal entries of A must be nonzero in order for M to 
be invertible. 


“ 

< 

* 

* 

* 



- 


0 

0 

0 


* 



* 

* 



* 

*s 

'Q 

0 

0 


* 

* 


K 

* 


M = 

* 

* 

hN 


0 


* 

* 

* 





* 

* 

* 




* 

* 

* 

* 




* 

* 

* 

* 

is 



Figure 01:The Lower Triangular Part of A 


Example 3 

Apply the Gauss - Seidel method to the system in Example 1 with 
x <0> =( 0 , 0 , 0 ) and six iterations. 

10 xi + X 2 - X 3 = 18 
XI + 15X2 + X3 = -12 

-xi + X 2 + 20x3 = 12 ( 6 ) 


Solution 
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*1 Tl 

For some k, let x (k) = x 2 =(xi, x 2 , x 3 ) and x (k + 1) = y 2 =(yi, y 2 > y 3 ) 

x A L v 3_ 

Again, firstly we construct matrices M and N from the coefficient matrix A. 

"10 1 -1 
Here ^4=1 15 1 

-1 1 20 
Since matrix M is constructed by 

1) taking the values along the diagonal and below the diagonal of coefficient 
matrix A. 

2) putting the zeros above the diagonal at upper trianular position. 

So 

"10 0 0 ' 

M = 1 15 0 

-1 1 20 

Now, 

"10 0 Ol [10 1 -ll [0 -1 1 

N = M-A= 1 15 0 - 1 15 1 = 0 0 -1 

-1 1 20 J [-1 1 20 J [0 0 0 

Now the recursion: Mx <k+l) = (M - A)x (k) +b ( here k = 0,1,...6) 

implies 

10 0 0 ][y 1 ] [0 -1 lir^l [18 

1 15 0 y 2 = 0 0 -1 x 2 + -12 

-1 1 2 oJ[_y 3 J [0 0 oJ[x 3 J L I? 

10jj 0 0 -x,+x 3 18 

=> y 1 15y 2 0 = -x 3 + -12 

_-y, y 2 2 °y 2 _ _ 0 _ 17 

10 Vj 0 0 -x 2 +x 3 +18 

^ y, 15y 2 0 = -x 3 -12 

_-y, y 2 2 °y 2 _ _ 0+17 

Comparing the corresponding entries on both sides, we have 


10yi = -X2 + X3 + 18 
yi+ 15 y 2 = -x 3 - 12 
-yi +y2 + 2 0 y 3 =17 

This further implies as 


©Virtual University Of Pakistan 


197 




16-Iterative Solutions of Linear Systems 


VU 


10vj = -x { -x 3 +18 => y 1 = (-x 2 + x 3 +18)/10 
y l +\5y 2 = -x 3 -12 => y 2 = (~y l -x 3 —12)/15 ► 
-y l +y 2 + 20y 3 =l7 ^ y 3 =( yi -y 2 +n)/20 


(7) 


Another way to view (7) is to solve each equation in (6) for xj, x?, x 3 , respectively and 
regard the highlighted x ’s as the values: 

Xi — (-X 2 + X 3 + 18)/10 

X 2 = (-X 1 —X 3 — 12)/15 

xj = (xj - xj + 17)/20 (8) 


Use the first equation to calculate the new x/ [called v/ in (7)] from x? and X 3 . Then, use 
this new xi along with xj in the second equation to compute the new X 2 . Finally, in the 
third equation, use the new values for xj and x? to compute x 3 . In this way, the latest 
information about the variables is used to compute new values. [A computer program 
would use statements corresponding to the equations in (8).] 


Fromx (0> = (0, 0, 0), we obtain 


x; = [-(0) + (0)+18]/10=18 

r—--- 

x 2 = [ - (1.8) 




(())- 12]/15 = -.9,2 


x 3 = [+(1.8) - (-.92) + 17]/20 = .986 


Thus, x n> = (1.8, -.92, .986). The entries inx (1> are used in (8) to produce x (2) and so on. 
Here are the MATLAB calculations reported to four decimal places: 


x (0) 

x (1) 

x (2) 

x (3) 

x (4) 

x <5) 

x (6) 

"O' 


" 1.8 ' 


"1.9906" 


"1.9998" 


" 2.0000 ' 


" 2.0000 ' 


" 2.0000 " 

0 


-.92 


-.9984 


-.9999 


-1.0000 


-1.0000 


-1.0000 

0 


.986 


.9995 


1.0000 


1.0000 


1.0000 


1.0000 


Observe that when k is 4, the entries in x (4> and x (k !> differ by less than .001. The values 
inx <6> in this case happen to be accurate to eight decimal places. 

Alternative Approach 

If we express the above system as 

10 xi + x 2 - X 3 — 18 Xi = (- X 2 + X 3 + 18)/10 - (a) 

Xi + 15x2+ x 3 = -12 =^> X 2 = (-X]—x 3 —12)/15 - (b) 

-x 1 + x 2 + 20 x 3 = 17^ x 3 = (xj -X 2 + 17)/20 - (c) 


1st Iteration 

Put x 2 =x 3 =0 in (a) 


©Virtual University Of Pakistan 


198 








16-Iterative Solutions of Linear Systems 


VU 


*,= 18/10=1.80 
Put xi=l .80 and x 3 =0 in (b) 
x 2 = (-1.80-0-12)/15= -0.92 
Put Xj=1.80, x 2 = -0.92 in (c) 
* 3 =(1.80+0.92+17)/20 = 0.9863 


xl 


~ 1.8 ~ 

x2 

= 

-.92 

x3 


.986 


2nd iteration 

Putx 2 = -0.92, x3=0.9863 in (a) 

*i=(0.92+0.9863+18)/10 = 1.9906 

Put Xi=1.9906(from 2 nd iteration) and x 3 =0.9863(from 1 st iteration) in (b) 
x 2 =(-l.9906-0.9863-12)/15 = -0.9984 

Put *1=1.9906, *2=-0.9984(both from 2 nd iteration) in (c) 
x 3 =(l.9906+0.9984+17)720 = 0.9995 


xl 


"1.9906" 

x2 

= 

-.9984 

x3 


.9995 


So in the similar fashion, we can get the next approximate solutions: x (3) ,x (4) ,x (5) and x (6) 
Next example will be solved by following this approach. 


Example 4 

Use Gauss-Seidel to approximate the solution of the linear system in example 2 to four 
decimal places. 

Solution 

As before, we will take x t = x 2 = x 3 = 0 as the initial approximation. First, we will 
substitute x 2 = 0 and x 3 = 0 into the right side of the first equation of (4) to obtain the new 
xi. Then, we will substitute x 3 = 0 and the new xi into the right side of the second 
equation to obtain the new x 2 , and finally, we will substitute the new xi and new x 2 into 
the right side of the third equation to obtain the new x 3 . The computations are as follows: 
x, = 0.85 - (0.05)(0) + (0.05)(0) = 0.85 

x 2 = -1.3 + (0.1)(0.85) + (0.1)(0) = -1.215 


x 3 = 1.8 + (0.1)(0.85) - (0.1X-1.215) = 2.0065 


Thus, the first Gauss-Seidel iterate is 


y i = 


0.8500 

-1.2150 

2.0065 


Similarly, the computations for second iterate are 
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x, = 0.85 - (0.05)(—1.215) + (0.05)(2.0065) = 1.011075 
x 2 = -1.3 +(0.1)(1.011075) + (0.1)(2.0065) = -0.9982425 
x 3 = 1.8 + (0.1)(1.011075) - (0.1)(—0.9982425) = 2.00093175 


Thus, the second Gauss-Seidel iterate to four decimal places is 


JT ~ 


1.0111 

-0.9982 

2.0009 


The following table shows the first four Gauss-Seidel iterates to four decimal places. 
Comparing both tables, we see that the Gauss-Seidel method produced the solution to 
four decimal places in four iterations whereas, the Jacobi method required six. 



yo 

yi 

yi 

ys 

y4 

Xl 

0 

0.8500 

1.0111 

1.0000 

1.0000 

X2 

0 

-1.2150 

-0.9982 

-0.9999 

- 1.0000 

X3 

0 

2.0065 

2.0009 

2.0000 

2.0000 


Comparison of Jacobi’s and Gauss-Seidel method 

There exist examples where Jacobi’s method is faster than the Gauss-Seidel 
method, but usually a Gauss-Seidel sequence converges faster (means to say iterative 
solution approaches to the unique solution), as in Example 2. (If parallel processing is 
available, Jacobi might be faster because the entries in x ! can be computed 
simultaneously.) There are also examples where one or both methods fail to produce a 
convergent sequence, and other examples where a sequence is convergent, but converges 
too slowly for practical use. 

Condition for the Convergence of both Iterative Mthods 

Fortunately, there is a simple condition that guarantees (but is not essential for) 
the convergence of both Jacobi and Gauss-Seidel sequences. This condition is often 
satisfied, for instance, in large-scale systems that can occur during numerical solutions of 
partial differential equations (such as Laplace’s equation for steady-state heat flow). 

An n x n matrix A is said to be strictly diagonally dominant if the absolute value of 
each diagonal entry exceeds the sum of the absolute values of the other entries in the 
same row. 

In this case it can be shown that A is invertible and that both the Jacobi and Gauss-Seidel 
sequences converge to the unique solution of Ax = b, for any initial x (0) . (The speed of 
the convergence depends on how much the diagonal entries dominate the corresponding 
row sums.) 

The coefficient matrices in Examples 1 and 2 are strictly diagonally dominant, but the 
following matrix is not. Examine each row: 
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-6 2 -3 
1 4 -2 
3-5 8 

The problem lies in the third row, because |8| is not larger than the sum of the 
magnitudes of the other entries. 

Note 

The practice problem below suggests a TRICK(rearrangement of the system of 
equations) that sometimes works when a system is not strictly diagonally dominant. 

Example 5 

Show that the Gauss-Seidel method will produce a sequence converging to the 
solution of the following systems, provided the equations are arranged properly: 

XI - 3x2 + X 3 = -2 
- 6 xi + 4 X 2 + 11 x 3 = 1 
5xi - 2 x 2 - 2 x 3 = 9 

Solution 

The system is not strictly diagonally dominant, as for the 1 st row 
\coefficient of x x | < | coefficient of x 2 1 +1 coefficient of x 3 1 

or |l|<|-3| + |l| 

so neither Jacobi nor Gauss- Seidel is guaranteed to work. In fact, both iterative methods 
produce sequences that fail to converge, even though the system has the unique solution 
xi = 3, X 2 = 2, X 3 = 1. However, the equations can be rearranged as 
5xj - 2x2 ~ 2x3 = 9 
XI - 3 x 2 + X 3 = -2 

-6x1 + 4X2 + 11x3 - 1 
So, 

for 1 st equation (row); 

| coefficient of x x \ > \coefficient of x 2 | + | coefficient of x 3 | 
or |5| > |-2| + |-2| 
for 2 nd equation(row); 

| coefficient of x 2 1 > | coefficient of Xj| + | coefficient of x 3 | 

or |-3| >|l| + |l| 

for 3 ld equation(row); 

| coefficient o/x 3 | > | coefficient of x 3 1 +1 coefficient of x 2 | 
or |ll| > |—6| + |4| 

Now the coefficient matrix is strictly diagonally dominant, so we know Gauss-Seidel 
works with any initial vector. In fact, if x ((>> = 0, then x H> = (2.9987, 1.9992, .9996). 


— 6 \ > | 2 | + |—3 
|4| > |1| + |-2| 
|8| = |3| + |—5| 
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Exercises 


Solve the system in exercise 1 to 3 using Jacobi’s method, with x <01 = 0 and three 
iterations. Repeat the iterations until two successive approximations agree within a 
tolerance of .001 in each entry. 


4x, + x, = 7 

1 . 1 2 

— Xj + 5x 2 = -7 


10xj-x 2 =25 
Xj + 8x 2 = 43 


3xj + x 2 =11 
3. -Xj -5 x 2 + 2x 3 =15 
3x 2 + 7x 3 = 17 


50xj - x 2 =149 
4. Xj -100x 2 +2x 3 = -101 
2x 2 +50x 3 = -98 


In exercises 5 to 8, use the Gauss Seidel method, with x (0) = 0 and two iterations. 
Compare the number of iterations needed by Gauss Seidel and Jacobi to make two 
successive approximations agree within a tolerance of .001. 


5. The system in exercise 1 
7. The system in exercise 3 


6. The system in exercise 2 
8. The system in exercise 4 


Determine which of the matrices in exercises 9 and 10 are strictly diagonally dominant. 


9. (a) 


5 4 
4 3 


(b) 


9 

5 

-2 


-5 2 

-8 -1 

1 4 


10.(a) 


-2 

3 


(b) 


3 

6 

-4 


1 

-4 

7 


Show that the Gauss Seidel method will produce a sequence converging to the solution of 
the following system, provided the equations are arranged properly: 


11. -6xj + 4 x 2 + 1 1x 3 =1 
5 Xj - 2x 2 - 2x 3 = 9 


-Xj + 4 x 2 - x 3 =3 


12. 4xj - x 2 


= 10 


- x 2 + 4x 3 = 6 
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Lecture 17 

Introduction to Determinant 


In algebra, the determinant is a special number associated with any square matrix. 
As we have studied in earlier classes, that the determinant of 2 x 2 matrix is defined as 
the product of the entries on the main diagonal minus the product of the entries off the 
main diagonal. The detenninant of a matrix A is denoted by det (A) or |A| 

a b 

For example: A = 

c d 

Then det (A) = ad-bc. 

or |A| = ad - be 

Example Find the determinant of the matrix A = 

1 2 

= Ix4-2x3 =4-6=-2 

3 4 

To extend the definition of the dcl(A ) to matrices of higher order, we will use subscripted 
entries for A 

a \\ a \2 
b 2\ b 22_ 

ci\\ #12 

det (A) = = a\ \b 22 ~ a\2 b 2 1 

b 2l b 22 

This is called a 2x2 determinant. 

The determinant of a 3x3 matrix is also called a 3x3 determinant is defined by the 
following formula. 

a ll a 12 a l 3 

a 21 a 22 a 23 

a 31 a 32 a 33 






For finding the determinant of the 3x3 matrix, we look at the following diagram: 
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We write 1 st and 2 nd columns again beside the determinant. The first arrow goes from 
an to a 33i which gives us product: a; ici22 a 33 • The second arrow goes from a n to a 3 i, 
which gives us product: aj2 a 23 a 31 • The third arrow goes from a [3 to a 3 2 , which gives us 
the product: a; ja 2 ja 32 • These values are taken with positive signs. 

The same method is used for the next three arrows that go from right to left downwards, 
but these products are taken as negative signs. 


“ a ll a 22 a 33+ a 12 a 23 a 31+ a 13 a 21 a 32 ~ a 13 a 22 a 31 ~ a ll a 23 a 32 ~ a 12 a 21 a 33 


Example 2 


Find the detenninant of the matrix A = 


1 

-4 

7 


2 3 
5 6 
-8 9 


detA = 


1 

-4 

7 


2 3 

5 6 

-8 9 


1 2 
-4 5 

7 -8 


3 

6 

9 


1 2 
-4 5 
7 -8 


= 1x5x9 + 2x6x7 + 3x(-4)x(-8) - 3x5x7 - lx6x(-8) - 2x(-4)x9 
-45+84 + 96-105+48 + 72 
= 240 

We saw earlier that a2x2 matrix is invertible if and only if its detenninant is nonzero. 
In simple words, a matrix has its inverse if its determinant is nonzero. To extend this 
useful fact to larger matrices, we need a definition for the determinant of the n x n matrix. 
We can discover the definition for the 3x3 case by watching what happens when an 
invertible 3x3 matrix A is row reduced. 


Gauss’ algorithm for evaluation of determinants 


1) Firstly, we apply it for 2x2 matrix say 


A = 


2 3 
4 3 


R' 2 —» R 2 -2R t (Multiplying 1 st row by 2 and then subtracting from 2 nd row) 


2 3 


1 

<N 

1 _ 

4-2(2) 3 — 2(3) J 


l 

m 

1 

O 

_ 1 


Now the determinant of this upper triangular matrix is the product of its entries on main 
diagonal that is 
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Det(A) = 2(-3) - 0 x 3 
2) For 3x3 matrix say 


B = 


-2 2 

-1 1 

2 0 


-3 

3 

-1 


- 6-0 


= -6 


By R[ 2 —» R 2] (Interchanging of 1 st and 2 nd rows) 


-1 1 3 

-2 2 -3 
2 0-1 


R' 2 —> R 2 -2/?, (Multiplying 1 st row by ‘-2’ and then adding in the 2 nd row) 

R' } —> R : , +2R { (Multiplying 1 st row by ‘2’ and then adding in the 3 rd row) 

"-1 1 3 " 

-00-9 
0 2 5 

By R 23 —» R, 2 (Interchanging of 2 nd and 3 ld rows) 

"-1 1 3 " 

-025 
0 0-9 

Now the determinant of this upper triangular matrix is the product of its entries on main 
diagonal and that is 
Det(B) = (-1) • 2 • (-9) = 18 
So in general, 

For a lx 1 matrix 

say, A = [a iy ] - we define det A = a n . 


For 2x2matrix 


j 2 

£Z 2 i Cl 2 2 


By R' 2 —» R 2 


f a A 

u 2\ 

\ a u J 


R 


provided that a n ^ 0 


n 12 


0 a --2I a 

u 22 m 12 


A = det ^4 = product of the diagonal entries 
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= a. 


f a ' 

a 22 — —a u 

V a n J 


a n®22 a \2®2\ 


For 3x3 matrix say 


CL i j ^12 ^13 


^3i ^32 ^33 


By R 2 - 


( a ^ 
u 2\ 

v«n y 


j ^ 3 —> ^3 _ 


“31 

v«n y 


/?, provided that a n ^ 0 


0 

0 


*12 


^22^11 ^12^21 ^23^11 ^13^21 


^32^11 ^12^31 ^11^33 ^13^31 


By i ? 3 —> i ? 3 - 


a 22 a \\ a \2 a 2l 


"n y 


i ? 7 provided that 


a 22 fl ll ^12^21 Q 


Q @22® U a ! 2 a 21 


0 0 


a 22 a \\ @\2,®2\ 


\ u n y 


@22@\\ a \2 a 2\ ^ 


v y 


v a n ^ 0 


Which is in echelon form.Now, 

A = det A = product of the diagonal entries 
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f _ V 

$ 22 ^q 1 ^ 12^21 

v y 

f 


$ n $ 33 $i3$3i 


(< 2 2 2^11 ^ 12^21 ) 


v w n y 


v a n y 
f 


W 


\ y a 22 a ll ^ 12^21 y y 


(# 22^11 ^ 12 ^ 2 l) 


a 22 a \\ a \2 a 2\ 


\ “n y 


\ a 22 a U ®\2®2\ J 


{( a 22 °ll a i2 a 2l)(^ll fl 33 ^13 fl 3l) (^23^11 a \3 a 2\ )( a 32°ll 0 12 fl 3l)} 

a \ I 

1(2 2 

j |^22^33 |^223^31 2^^21 |$33 I 2^^21 3^^3 ] $23I $32 ^ ^^23 l2 ^^31 ^ 3 d^1 ^*32 ^ 11 


2 2 
I $22$33 $i j$22$23^^3 1 2^*21 1 ^^33 $^23^d 2$^32 ^ ^*23^d I j2^^3 ^ ^ 3 d ^ j ^*32^d j 


$1 j r 1 

— (^11^22^33 — a 22 a \i a 2\ ~ a \2 a 2\ a 33 — ^23^11^32 ^23^12^31 "*“^13^21^32/ 


$ 11 ^ 22^33 ^ ^ 12 ^ 23^31 ^ ^* 13 ^* 21^*32 ^* 12 ^* 21^*33 ^* 11 ^* 23^*32 ^* 13 ^* 22^*31 


Since ^4 is invertible, A must be nonzero. The converse is true as well. 


To generalize the definition of the determinant to larger matrices, we will use 
2x2 determinants to rewrite the 3x3 determinant A described above. Since the 
terms in A can be grouped as: 

/\ 1 a 11 tt,, a 11 tt 1 1 tt |, tt, ^ tt 2 ^ t/1, tt, | a 22 1 l 1 tt 2 2 a ., 3 tt 2, tt 2 2 ci ,, a 3 3 1 

tt 1 ] 1 tt,, a 22 ci ,2 ci 2") 1 ci 2,^ ci , 2 ci 22 ci ,2 ci 2 2 1 i ci 1 ^ 1 ci , 2 ci 2, ci ,, a 3 3 1 


A = tfjj - det 

a 22 

a 23 

- a n • det 

1 

& 

to 

a 22 

+ a 13 • det 

^21 ^22 


_ a 22 

®33_ 


L^31 

a 2i'_ 


_ a 3 1 a 32_ 


Cf 

II 

<1 

a 22 

a 23 


a 2\ 

a 23 

+ <7j 3 ’ 

a 2\ 

a 2 2 



a \2 






a 32 

a 23 


a 2\ 

U 33 


a 3\ 

a 32 


For brevity, we write A = a n ■ det A n - a l2 ■ det A u + a n ■ det d 13 (3) 


det( 4 i) = 


a i2 a i2 


det(4 2 ) = 


a 2 1 a 22 


$31 $^33 


and det(d 13 ) = 


$3 j $32 


where 

A n is obtained from A by deleting the first row and first column. 

A 12 is obtained from A by deleting the first row and second column. 

An is obtained from A by deleting the first row and third column. 

So in general, for any square matrix A, let Ay denote the sub-matrix formed by deleting 

the ith row and jth column of A. 


Let’s understand it with the help of an example. 
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Example3 

Find the determinant of the matrix A = 


1 

5 

3 


4 3 
2 4 
6 3 


Solution Given 


A = 


1 4 3 
5 2 4 
3 6 3 


A\ = 


1 4 3 
5 2 4 
3 6 3 


2 4 


5 4 


5 2 


- 4 


+ 3 


6 3 


3 3 


3 6 


= 1 


= 1(2x3-4x6)-4(5x3-4x3) + 3(5x6-2x3) 


= 1(6 - 24) - 4(15 -12) + 3(30 - 6) 
= 1(-18)-4(3)+ 3(24) 

= -18-12 + 72 


= 42 


For instance, if 


A 


1-250 

2 0 4 -1 

3 10 7 

0 4-20 


then A 32 is obtained by crossing out row 3 and column 2, 


“l 

-2 

5 

0 “ 

2 

0 

4 

-1 

3 

1 

0 

7 

0 

4 

-2 

0 


so that 


^32 


1 

2 

0 


5 

4 

-2 


0 

-1 

0 


We can now give a recursive definition of a determinant. 

When n = 3, det A is defined using detenninants of the 2x2 submatrices A Xj . 

When n = 4 , del A uses determinants of the3x3 submatrices A Xj 

In general, an nxn detenninant is defined by determinants of (n - 1) x (n - 1) sub matrices. 
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Definition 

For n > 2, the determinant of nxn matrix A = [a..] is the sum of n tenns of the form 
±a lj x(det A ly ), with plus and minus signs alternating, where the entries a n ,a l2 ,--,a hl 
are from the first row of A. 

Here for a ij , 

i = 1,2,3,..., n (1 <i<n) 
j = 1,2,3,...,/! (1 <j<ri) 

n 

In symbols, det A = a n det A n -a 12 det A n +.... +(-1 ) l+ "a hl det A ln = ^(-l) 1+ -'a ly . det A Xj 

7=1 


Example 4 

Compute the determinant of A = 


1 

2 

0 


5 

4 

-2 


0 

-1 

0 


Solution 

Here A is nxn = 3x3matrix such that 
/ = 1,2,3 
J =1,2,3 

n 

v det(^4) = ^(-1 ) x+j a ij det A Xj and here j = 1,2,3 

7=1 

3 

det(A) = ^(-1 ) Uj a Xj det A Xj = (-l) 1+1 o n det A : , +(-l) 1+2 fif 12 detA 12 +(-l) 1+3 o 13 det A 13 

7=1 

= a l j det A u -a n det A n + a u det A n 



' 4 -ll 


'2 

-ll 


'2 

4' 

det A = 1. det 

-2 0 

-5.det 

0 

0 

+ 0. det 

0 

-2 


det A = 1. 


0 


5. 


2 -1 

0 0 


0 . 


4 

-2 


= 1 [4(0) - (-l)(-2)] -5 [ 2(0) - 0(-1)] +0[2(-2) - 4(0)] 


= 1(0 - 2) - 5(0 - 0) + 0(-4 - 0) = -2 


Minor of an element 

If A is a square matrix, then the Minor of entry aij (called the ijth minor of A) is 
denoted by Mj and is defined to be the determinant of the sub matrix that remains when 
the ith row and jth column of A are deleted. 
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In the above example, Minors are as follows: 


M u = 


4 

-1 

JVC = 

2 

-1 

, A/,, = 

2 

4 

-2 

0 

, 12 

0 

0 

, ju 13 

0 

-2 


Cofactor of an element 

The number Cij=(-l) l+j Mij is called the cofactor of entry ay (or the ijth cofactor of A). 
When the + or - sign is attached to the Minor, then Minor becomes a cofactor. 

In the above example, following are the Cofactors: 


C„ =(-l) 1+1 M,„ C l2 =(—1) 1+2 M 


c , 3 =(—1 ) M M 


12 ’ 


13 


C u =(-l) 


1+1 


4 -1 

-2 0 


C 12 =(- 1) 


1+2 


2 -1 

0 0 


C 13 =(-l) 


1+3 


2 

0 


Example 5 Find the minor and the cofactor of the matrix A = 


3 1 -4 

2 5 6 

1 4 8 


Solution Here 


A = 


3 1 -4 

2 5 6 

1 4 8 


The minor of entry a 11 is 


M n = 


1$ 5 6 

4 8 

and the corresponding cofactor is 


3 1 -4 


5 6 
4 8 


= 5x8-6x4 = 40 - 24 = 16 


C u =(-\y +l M u =M u =\6 
The minor of entry a 32 is 


m 32 = 


3 : 

-4 





3 -4 

2 i 

i 6 

= 





2 6 


i—8- 




= 26 


and the corresponding cofactor is 

C3 2 =(-1) 3+2 M 32 =-M 32 =- 


3 -4 
2 6 


= -26 
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Alternate Definition 

Given A = [a..], the (i, j)-cofactor of A is the number C ij given by 

C ; ,=(-l)^'det4 (4) 

Then det A = a n C n +a n C n +.... + a ln C ln 

This formula is called the cofactor expansion across the first row of A. 


Example 6 Expand a 3x3 detenninant using cofactor concept A = 
Solution Using cofactor expansion along the first column; 


1 

-4 

7 


2 3 
5 6 
-8 9 


1 2 3 

-4 5 6 

7-8 9 


(1)(-1) 1+1 


5 

-8 


6 

9 


(— 4 )(— 1) 2+1 


2 

-8 


3 

9 


(7)(—1) 3+1 


3 

6 


Now if we compare it with the formula {A), 


= 1 C u + (—4) C 21 +7 C 31 


= (!)(-1) 2 


5 6 
-8 9 


+ (—4)(—l) 3 


2 3 
-8 9 


(7)(—l) 4 


2 3 
5 6 


= ( 1 )( 1 ) 


5 6 
-8 9 


+ (—4)(—1) 


2 3 
-8 9 


(7)(1) 


2 3 
5 6 


= 1 


5 6 
-8 9 


2 3 
-8 9 


+ 7 


2 3 
5 6 


= 1(45 —(—48)) + 4(18 — (—24)) +7(12 - 15) 


= 1(45 + 48) + 4 (18 + 24) + 7 (12-15) 


= (1)(93) + (4)(42) + (7)(—3) = 240 


Using cofactor expansion along the second column, 
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1 

-4 

7 


2 3 
5 6 
-8 9 


= (2)(-l) 


1+2 


-4 6 
7 9 


+ (5)(-l) 


2+2 


1 3 
7 9 


+ (-8)(-l) 


3+2 


1 

-4 


= (2)(-l) 3 


-4 6 
7 9 


+ (5)(-l) 4 


1 3 
7 9 


+ (-8)(-l) 5 


1 3 
-4 6 


= (2)(-l) 


-4 6 
7 9 


+ (5)(1) 


1 3 
7 9 


+ (-8)(-l) 


1 3 
-4 6 


= -2 


-4 6 
7 9 


+ 5 


1 3 
7 9 


+ 8 


1 3 
-4 6 


3 

6 


= -2(-36 — 42 ) + 5(9 — 21) + 8(6 — ( — 12)) 

= (—2)(—78) + (5)(—12) + (8)(18) = 240 

Theorem 1 The determinant of an n x n matrix A can be computed by a cofactor 
expansion across any row or down any column. The expansion across the ith row using 
the cofactors in (4) is 

det A = a n C n + ci n C n + • • • + u C 1 

il il iZ iZ in in 

The cofactor expansion down the jth column is 

det A — a lj C 1J + a 2J C 2 j -t a n jC n j 


The plus or minus sign in the (i, j)-cofactor depends on the position of a tj in the matrix, 
regardless of the sign of a tj itself. The factor (-l)' +; determines the following 
checkerboard pattern of signs: 

— + .... 

+ — 

— + 


+ 


+ 
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Example 7 Use a cofactor expansion across the third row to compute del A, where 


A = 


1 

2 

0 


5 

4 

-2 


0 

-1 

0 


Solution Compute det A = a 31 C 31 + a 32 C 32 + a 33 C 33 


= (-1) 3+1 <7 31 det^l 31 + (-l) 3+ 'a 32 det^l 32 +(-l) 3+3 a 33 det^4 33 


= 0 


5 

4 


0 

-1 



0 

-1 


+ 0 


1 5 

2 4 


= 0 + 2(-l) + 0 = -2 


Theorem 1 is helpful for computing the determinant of a matrix that contains many 
zeros. For example, if a row is mostly zeros, then the cofactor expansion across that row 
has many terms that are zero, and the cofactors in those terms need not be calculated. 

The same approach works with a column that contains many zeros. 


Example 8 


Evaluate the determinant of A = 


2 0 
-1 2 
3 0 

8 6 


0 

4 

0 

0 


5 

1 

3 

0 


Solution 


det(yl) = 


2 

-1 

3 

8 


0 

2 

0 

6 


0 

4 

0 

0 


5 

1 

3 

0 


Expand from third column 
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det(/4) = 0 x C 13 + 4 x C 23 + 0 x C 33 + 0 x C 43 
= 0 + 4 x C 23 + 0 + 0 
= 4xC 23 

2 0 5 

= 4x(-l) 2+3 3 0 3 

8 6 0 

Expand from second column 

f 2 5 ^ 

= -4 0 + 0 + (-6) 

3 3 

V j j j 

2 5 

= <-4) <- 6 ) 3 3 
= -216 

Example 9 Show that the value of the determinant is independent of 0 

sin 0 cos 6 0 

A= —cos 0 sin# 0 

cos 6- sin 6 sin 9 + cos 6 1 

sin# cos# 0 

Solution Consider A = -cos# sin(9 0 

cos 9- sin 6 sin 6 + cos 6 1 

Expand the given determinant from 3 rd column we have 

= 0 - 0 + (-l) 3+3 [sin 2 6 + cos 2 6 ] = 1 
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Example 10 Compute det ,4, where A = 


3 

0 

0 

0 

0 


-7 

2 

0 

0 

0 


8 9-6 

-5 7 3 

1 5 0 

2 4-1 
0-2 0 


Solution The cofactor expansion down the first column of A has all terms equal to zero 
except the first. 


Thus 


det A = 3 


2 

0 

0 

0 


-5 

1 

2 

0 


7 

5 

4 

-2 


3 

0 

-1 

0 


0.C 21 +0.C 31 -0.C 41 +0.C 51 


Henceforth, we will omit the zero terms in the cofactor expansion. 


Next, expand this 4x4 detenninant down the first column, in order to take advantage of 
the zeros there. 


1 5 0 

We have det A = 3x2 2 4 -1| 

0-2 0 

This 3x3 determinant was computed above and found to equal -2. 


Hence, det.4 = 3x2x(-2) = -12 . 


The matrix in this example was nearly triangular. The method in that example is easily 
adapted to prove the following theorem. 


Triangular Matrix 


A triangular matrix is a special kind ofmxn matrix where the entries either below or 
above the main diagonal are zero. 


"1 

4 

2 ' 


"1 

0 

O' 

0 

3 

4 


2 

8 

0 

0 

0 

1 

is upper triangular and 25 x 25 

4 

9 

7 


J is lower triangular matrices. 


Determinants of Triangular Matrices 

Determinants of the triangular matrices are also easy to evaluate regardless of size. 
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Theorem If A is triangular matrix, then det (A) is the product of the entries on the main 
diagonal. 


Consider a 4 x 4 lower triangular matrix. 



a u 

0 

0 

0 

A = 

a 2l 

a 22 

0 

0 


«31 

a 32 

a 33 

0 


_«41 

a 42 

a 43 

a 44 


Keeping in mind that an elementary product must have exactly one factor from each row 
and one factor from each column, the only elementary product that does not have one of 
the six zeros as a factor is (a n a 22 a i3 a 44 ) . The column indices of this elementary product 
are in natural order, so the associated signed elementary product takes a +. 


Thus, 


det (A)= a n x a 22 x a 33 x a 44 


Example 11 


-2 

0 

0 


5 7 
3 8 
0 5 


= (-2X3X5) = -30 


1 0 0 
4 9 0 

-7 6 -1 
3 8-5 


0 

0 

0 

-2 


- (1)(9)(—1)(—2) = 18 


1 2 7 

0 1 -4 

0 0 2 

0 0 0 


-3 

1 

7 

3 


= (1)(1)(2)(3) = 6 


The strategy in the above Example of looking for zeros works extremely well when an 
entire row or column consists of zeros. In such a case, the cofactor expansion along such 
a row or column is a sum of zeros. So, the determinant is zero. Unfortunately, most 
cofactor expansions are not so quickly evaluated. 

Numerical Note By today’s standards, a 25x25 matrix is small. Yet it would be 
impossible to calculate a 25x25 determinant by cofactor expansion. In general, a cofactor 
expansion requires over n! multiplications, and 25! ~ 1.5x10 25 . 
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If a supercomputer could make one trillion multiplications per second, it would have to 
run for over 500,000 years to compute a 25x25 determinant by this method. 
Fortunately, there are faster methods, as we’ll soon discover. 


Example 12 Compute 


5 

0 

-5 

0 


-7 2 2 

3 0-4 

-8 0 3 

5 0-6 


Solution Take advantage of the zeros. Begin with a cofactor expansion down the third 
column to obtain a 3 x 3 matrix, which may be evaluated by an expansion down its first 
column, 


5-7 2 

0 3 0 

-5 -8 0 

0 5 0 



0 

3 

-4 

II 

T 

+ 

-5 

-8 

3 


0 

5 

-6 


= 2 • (— 1 ) 2 + 1 (— 5 ) 


3 

5 


-4 

-6 


= 20 


The -1 in the next-to-last calculation came from the position of the -5 in the3x3 
determinant. 
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Exercises 

Compute the determinants in exercises 1 to 6 by cofactor expansions. At each step, 
choose a row or column that involves the least amount of computation. 


1 . 


6 0 0 5 

17 2-5 
2 0 0 0 
8 3 18 


2 . 


1- 252 
0 0 3 0 

2- 6-75 
5 0 4 4 


3. 


3 5-84 

0-2 3-7 

0 0 15 
0 0 0 2 


4. 


4 0 0 0 

7-100 
2 6 3 0 

5-8 4-3 


5. 


4 0 -7 3 -5 

0 0 2 0 0 

7 3 -6 4 -8 

5 0 5 2 -3 

0 0 9 -1 2 


6 . 


6 3 2 4 0 

9 0-410 

8-5671 

3 0 0 0 0 

4 2 3 2 0 


Use the method of Example 2 to compute the determinants in exercises 7 and 8. In 
exercises 9 to 11, compute the detenninant of elementary matrix. In exercises 12 and 13, 

a b 

verify that det EA = (det E ). (det A), where E is the elementary matrix and A = 

c d 



3 

0 

4 


2 

-4 

3 


"1 

0 

O' 


k 

0 

0 " 

7. 

2 

3 

2 

8 . 

3 

1 

2 

9. 

0 

1 

0 

10 . 

0 

1 

0 


0 

5 

-1 


1 

4 

-1 


0 

k 

1 


0 

0 

1 


"0 

1 

0 " 


















'1 

k 



"0 

1 ] 






11 . 

1 

0 

0 

12 . 

0 

1 


13. 

1 

0 _ 







0 

0 

1 














14. Let A = 


3 1 

4 2 


. Write 5 A. Is det 5A = 5 det A? 


15. Let A = 


a b 
c d 


and k be a scalar. Lind a formula that relates det (kA )to k and det A. 
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Lecture 18 


Properties of Determinants 

In this lecture, we will study the properties of the determinants. Some of them have 
already been discussed and you will be familiar with these. These properties become 
helpful, while computing the values of the determinants. The secret of determinants lies 
in how they change when row or column operations are performed. 

Theorem 3 (Row Operations): Let A be a square matrix. 

a. If a multiple of one row of A is added to another row, the resulting 
determinant will remain same. 

b. If two rows of A are interchanged to produce B, then det B = -det A. 

c. If one row of A is multiplied by k to produce B, then det B = k . det A. 


The following examples show how to use Theorem 3 to find determinants efficiently, 
a. If a multiple of one row of A is added to another row, the resulting determinant 
will remain same. 

Example 



a n 

a \2 

a xh 

A= 

a 2\ 

a 22 

a 23 


a 3 1 

a 32 

a 33 


Multiplying 2nd row by non-zero scalar say 'k'as 


ka 2 ! ka 22 ka 23 - adding this in \st row then 'A' becomes 


a x j + ka 


21 


a 


12 


ka 


22 


a 


13 


ka 


23 


a 


21 


a 


22 


a 


23 


a, 


a , 


a. 


R' —> R x + kR 2 


31 32 33 

If each element of any row(column) can be expressed as sum of two elements then the 
resulting determinant can be expressed as sum of two detenninants, so in this case 



a u 

a \2 

a i3 


ka 2l 

ka 22 

ka 23 

A = 

a 2\ 

a 22 

a 23 

+ 

a 2 \ 

U 22 

a 23 


a 3\ 

a 32 

a 33 


a 3\ 

a 32 

a 33 


a n 

a \2 

a \3 



a 2\ 

a 2 2 

a 23 


A = 

a 2 1 

a 22 

a 23 

+ k 

a 2\ 

a 2 2 

a 23 



a 3i 

a 32 

a 33 



a 3\ 

U 32 

a 33 



By using property (c) of above theorem 3. 


If any two rows or columns in a determinant are identical then value of this determinant 
is zero. So in this case R x = R 2 
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a n 

a \2 

a l3 

A = 

a 2\ 

a 22 

a 22> 


a 31 

a 32 

a 33 


a u 

a \2 

a U 

= 

a 2\ 

a 22 

a 23 


«31 

tt 22 

a 33 


+ Jfc(0) 

= A 


b. If two rows of A are interchanged to produce B, then det B = -det A. 


Example 1 


"1 2 3~ 

A- 5 1 1 

0 8 9 

1 2 3 

Now, det A = 5 1 1 = 1(9-8)-2(45-0)+ 3(40-0) = 1-90+ 120 = 31 

0 8 9 

"2 1 3 

Now interchange column 1st with 2 nd we get a new matrix, B = 1 5 1 

8 0 9 

2 1 3 

det 5 = 1 5 1 = 2(45 - 0) -1(9 - 8) + 3(0 - 40) = 90 -1 -120 = -31 

8 0 9 


c. If one row of A is multiplied by k to produce B, then det B = k . det A. 


"1 2 3" 

A= 5 0 1 
0 8 9 

\A\ = 1(0 - 8) - 2(45 - 0) + 3(40 - 0) 

= -8-90 + 120 = 22 
MultiplingR x by k, we get say 

'Ik 2k 3k 
B= 5 0 1 

0 8 9 
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\B\=k(40 - 0) - 2*(45 - 0) + 3*(40 - 0) 
= 40* -90* + 120*=22* 

=k\A\ 

Example 2 


Evaluate A = 


Solution 


12 3 4 

2 3 4 1 

3 4 12 

4 12 3 


det.4 = 


1 2 


12 3 4 

2 3 4 1 

3 4 12 

4 12 3 

3 4 

0 -1 -2 -7 

0 -2 -8 -10 
0 -7 -10 -13 


by R’ ~^R 2 + (-2 )R l , 7? 3 ' ~^R } + (-3)7?,, 7?/ 7? 4 + (-4)7?, 


-1 

-2 

-7 

-2 

-8 

-10 

-7 

-10 

-13 


expanding from 1st column 


= (— 1 )(— 2 )(— 1 ) 


= (- 2 ) 


- (-2) 

- (—2)(2)(—4) 


1 2 7 

1 4 5 

7 10 13 

1 2 7 

0 2-2 
0 -4 -36 

2 -2 


taking (-1), (-2) and (-1) common from 1st, 2nd, 3rd 


by R; ~^R 2 + (-1)7?,, 7?/ -> 7? 3 + (-7)7?, 


rows 


-4 -36 

1 -1 


= 16 
= 160 


1 -1 

0 10 


1 9 

by t? 2 + (-1)7?, 


expanding by 1st column 

taking 2 and (-4) common from 1st and 2nd rows respectively. 


VU 
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Example 3 Evaluate the determinant of the matrix A = 


4 2 
1 1 


3 

2 


5 

6 
0 
5 


10 

3 

5 

8 


Solution 


det.4 = 


4 2 
1 1 


1 1 
4 2 


3 

2 


3 

2 

6 

5 

0 

5 


5 

6 
0 
5 
3 

10 

5 

8 


10 

3 

5 

8 


interchanging R l and R 2 ( R [ 2 ) 


116 3 

0 -2 -19 -2 

0 -4 -42 -16 
0 2 5 8 

-2 -19 -2 

-4 -42 -16 
2 5 8 

2 19 2 

=(-l ) 3 4 42 16 
2 5 8 

2 19 2 

4 42 16 
2 5 8 

1 19 2 

= - 2 I 2 42 16 
1 5 8 

1 19 2 

= (-2) 0 4 12 

0 -14 6 


By R 2 ~^R 2 + (-4)R ] ,R 3 R 3 + (-7)R ] 


expanding from 1 st column 


taking (-l)asa common factor from Rj and R 2 


By R’ ~^R 2 + (-2 )R l , R 3 ~^R 3 + (-I)Rj 


vu 
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1 19 2 

= (-2)0 4 12 i? 2 +(-2)i? 1 ,i? 3 +(-l)i? 1 

0 -14 6 

4 12 

= -2 expand from 1st column 

-14 6 

= - 2(24+168)= - 3 84 



V 

a + x 

b + c 

Example 4 Without expansion, show that 

V 

b + x 

c + a 


V 

c + x 

a + b 


Solution 


X 

a + x 

b + c 

X 

b + x 

c + a 

X 

c + x 

a + b 


x a+x-x b + c 

= x b + x-x c + a By —» C 2 - C 1 

x c+x-x a+b 
x a b + c 
= x b c + a 
x c a + b 

Taking' x' common from C\ 

1 a b + c 
=x 1 b c+a 
1 c a + b 
1 a+b+c b+c 

=x 1 b + c + a c + a By —»C 2 + C 3 

1 c+a+b a+b 

Now taking (a+b+c) common form C 2 
1 1 b + c 

=x(a + b + c) 1 1 c + a 

1 1 a + b 

= 0 as column 1st and 2nd are identical (Cj = C,). So its value will be zero. 
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2 3 10 1 

113 12 

Example 5 Evaluate A = 2 1 2 3 4 

3 2 112 

4 110 0 

Solution Interchanging R i and AS, we get 
113 12 

2 3 10 1 

A=-2 1 2 3 4 

3 2 112 

4 110 0 

R 2 —> R 2 - 2 R t , /?,' i? 3 - 2R t , i? 4 ' R 4 - 3 R t , /?/ —> R s - 4 R t 

113 12 

0 1 -5-2 -3 

= -0-1-4 1 0 

0 -1 -8 -2 -4 

0 -3 -11 -4 -8 

expand from C| 

1 -5 -2 -3 

-1-41 0 

~~-l -8 -2 -4 

-3 -11 -4 -8 

R —> R, + R t , /?, —> /?, + /?!, /? 4 —> f? 4 + 3f?! 

1 -5 -2 -3 

0 -9 -1 -3 

0 -13 -4 -7 

0 -26 -10 -17 

expand from C| 

-9 -1 -3 

= --13 -4 -7 

-26 -10 -17 
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taking (-1) common from 1st, 2nd and 3rd row 
9 1 3 

= 13 4 7 

26 10 17 

interchange 1st and 2nd Column(C' 2 ) 

1 9 3 

= - 4 13 7 

10 26 17 

C[ —> C 2 - 9C,, C[ —> C 3 - 3(7, 

1 0 0 

= - 4 -23 -5 

10 -64 -13 


expand from 1st row 


-23 

-64 


-5 

-13 


= -(299-320) = 21 


An Algorithm to evaluate the determinant 

Algorithm means a sequence of a finite number of steps to get a desired result. The word 
Algorithm comes from the famous Muslim mathematician AL-Khwarizmi who invented 
the word algebra. 

The step-by-step evaluation of defiM) of order n is obtained as follows: 

Step 1: By an interchange of rows of A (and taking the resulting sign into account) bring 
a non zero entry to (1,1) the position (unless all the entries in the first column are zero in 
which case det^4=0). 

Step 2: By adding suitable multiples of the first row to all the other rows, reduce the 
(n-1) entries, except (1,1) in the first column, to 0. Expand det(A ) by its first column. 
Repeat this process or continue the following steps. 

Step 3: Repeat step 1 and step 2 with the last remaining rows concentrating on the second 
column. 

Step 4: Repeat step l,step2 and step 3 with the remaining (n-2) rows, (n-3) rows and so 
on, until a triangular matrix is obtained. 

Step5: Multiply all the diagonal entries of the resulting triangular matrix and then 
multiply it by its sign to get det(A) 
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1 -4 2 

-2 8 -9 

-17 0 

Solution The strategy is to reduce A to echelon form and then to use the fact that the 
determinant of a triangular matrix is the product of the diagonal entries. The first two 
row replacements in column 1 do not change the detenninant: 

1 -4 2 

det A= -2 8 -9 

-17 0 

1 -4 2 

0 0 —5 By R-, —> R 7 + 2 R l , —> R^ + R ' 3 

0 3 2 

An interchange of rows 2 and 3 (R 33 ), it reverses the sign of the detenninant, so 
1 -4 2 

det A = - 0 3 2 = —(1)(3)(—5) = 15 

0 0-5 

Example 7 Compute det A, where 
"2 -8 6 8 

3 -9 5 10 

A = 

-3 0 1-2 

1 -4 0 6 

Solution Taking’2’ common from 1 st row 
1-434 
3 -9 5 10 

det A = 2 

-3 0 1-2 

1-406 

1-434 
0 3-4-2, 

det A = 2 Bv R, —> R 1 - 3 R,, R, —> R, + 3 R, ,R, —> R, - R, 

0 0 -6 2 '- 2 1 3 3 1 4 4 1 

0 0-32 
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det A = 2 


1-434 
0 3-4-2 

0 0-62 
0 0 0 1 
= 2.{(l)(3)(-6)(l)} = -36 


(By K 


R 4 -±R 3 ) 


Example 8 Show that 


Solution 

x 2 2 2 

2x22 
2 2x2 

2 2 2 x 

i 

x + 6 222 
x+6 x 2 2 

x+6 2x2 
x+6 2 2 x 


2 

x 

2 

2 


2 

2 

x 

2 


(x + 6)(x-2) 3 


By C 1 '-»C 1 + (C 2 + C 3 +C 4 ) 


Taking (x+6) common from 1 st column 

|1 2 2 2 | 


= (x + 6) 


1x22 
12x2 
1 2 2 x 


(x + 6) 


12 2 2 

0 x-2 0 0 

0 0 x-2 0 

00 0 x-2 


By R 2 > R 2 ~ Ri> R 3 —■► R 3 ~ R\ i R4 R 4 ~~ R\ 


And this is the triangular matrix and its determinant is the prodcut of main diagonal’s 
entries. 

= (x + 6)(x - 2) 3 
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Example 9 Compute del A, where A = 


3-12-5 
0 5-3-6 

-6 7-7 4 

-5 -8 0 9 

-1 2 -5 

5 -3 -6 

5 -3 -6 

-8 0 9 


Solution A 


' 3 


0 

det A = 

0 


-5 

= 0 


Example 10 


' 0 


2 

A = 

0 


-2 - 

Solution 


3 -1 

0 5 

-6 7 

-5 -8 


R j —^ R j + 2i?j 


as R 2 = R 3 


A = 


1 

5 

3 


0 

2 

0 


2 

-7 

6 

4 

1 

5 
3 


-2 -5 


-1 

3 
2 

-2 

2 

-7 

6 

4 


-1 

3 

2 

-2 


2 

-3 

-7 

0 


-5 

-6 

4 

9 


i?, —> R , + i?. 


0 12-1 
2 5-73 
0 3 6 2 

0 0-31 
"2 12-1 
0 5-73 
0 3 6 2 

0 0-3 1 

Expanding from 1 st row and 1 st column 


= (-l) 


By R', 


12 
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= -2 
= (-2 



-7 3 

6 2 

-3 1 

+ 6) - (-7)(3 - 0) + 3(—9 - 0)} 


= 54 


Remarks 

Suppose that a square matrix A has been reduced to an echelon fonn U by row 
replacements and row interchanges. 

If there are r interchanges, then det( A ) = (-l) r det(77) 

Furthermore, all of the pivots are still visible in U (because they have not been scaled to 
ones). If A is invertible, then the pivots in U are on the diagonal (since A is row 
equivalent to the identity matrix). In this case, det V is the product of the pivots. If A is 
not invertible, then U has a row of zero and det U = 0. 



• 

o 

o o 


• 

o 

o 

o 


0 

• 

o o 


0 

• 

o 

o 

u = 

0 

0 

• ° 

u = 

0 

0 

• 

o 


0 

0 

0 • 


0 

0 

0 

0 


det U ^ 0 det U = 0 


Thus we have the following formula 

^ U-Vf .{Productof pivots inU) When Ais invertible 

[0 When A is not invertible 

Example 

Case-01 For 2x2 invertible matrix 

Reducing given 2x2 invertible matrix into Echelon form as follows; 

[4 5l 
A = 

_3 2_ 

By interchanging 1 st and 2 nd rows( R \ 2 ) 


3 2" 

4 5 _ 

v one replacement of rows has occurred, .'. r = 1 


3 2" 

7 

By R' —> R 1 - — R, , we have desired row-echelon form U = 

"3 2" 

7 

0 - 
3 

3 

0 - 
3 


Thus using the above fonnula as follows; 


( 1 ) 
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det yd = (-1)' .(Product of pivots inU) = (-1)'(3~) = -7 


Case-02 For 2 x 2 non-invertible matrix 
In this case say; 


A = 


4 5 

8 10 
4 5 
0 0 


By R 2 


R , 


— 2R { .desired row-echelon form is U = 


4 5 
0 0 


Here no interchange of rows has occurred. So, r = 0 and 
det A = (-iy .(Product of pivots inU) = (-1)°(4 -0) = 0 


Theorem 5 If A is an n x n matrix, then det A 1 = det A. 

1 4 1' 


Example 11 If yd = 


Now 


det yd = 


2 1 2 
3 1 3 

1 4 1 

2 1 2 
3 1 3 


, find detfd) and det (A ) 


= 1(3 - 2) - 4(6 - 6) +1(2 - 3) = 1 - 0 -1 = 0 


A‘ 


det ^4' = 


1 2 3 
4 1 1 

1 2 3 
1 2 3 
4 1 1 

1 2 3 


1(3-2)-2(12-1) + 3(8-1) = 1-22 + 21 = 0 


Remark 

Column operations are useful for both theoretical purposes and hand computations. 
However, for simplicity we’ll perform only row operations in numerical calculations. 


Theorem 6 (Multiplicative Property) 

If A and B are n x n matrices, then det(4/?) = (det .d)(det B ) . 


Example 12 Verify Theorem 6 for A = 


6 1 
3 2 


and B 


4 3 
1 2 
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Solution 


"6 

f 

4 3l 


"25 

20" 

_3 

2 

1 2 


14 

13 


and det AB = 25.13-20.14 = 325 - 280 = 45 


Since det A = 9 and det B = 5, (det 4)(det B ) = 9.5 = 45 = det AB 


Remark 

det (A + B) ^ det ,4 + det B, in general. 
For example, 


If A = 

A + B 


2 3 

1 - 5 
"0 O' 
0 0 


and B 


-2 -3 
-1 5 


=> det (A + B) = 0 


. Then 


det A + det B 


2 

1 



-3 

5 


= (-10-3) + (-10-3) = -26^det(^ + 5) 


vu 
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Exercise 


Find the detenninants in exercises 1 to 6 by row reduction to echelon fonn. 



1 

3 

0 

2 


1 

3 

3 

-4 


-2 

-5 

7 

4 


0 

1 

2 

-5 

1. 

3 

5 

2 

1 

2. 

2 

5 

4 

-3 


1 

-1 

2 

-3 


-3 

-7 

-5 

2 


3. 


1 -1-30 

0 15 4 
-12 8 5 
3 -1-2 3 


4. 


1 3-10-2 

0 2-4-1 -6 

-2 -6 2 3 9 

3 7 -3 8 -7 

3 5 5 2 7 


5. 


1-231 
5-963 
-1 2 -6 -2 
2 8 6 1 


6 . 


13 15 3 

-2 -7 0-4 2 
0 0 10 1 

0 0 2 1 1 

0 0 0 1 1 


Combine the methods of row reduction and cofactor expansion to compute the 
determinants in exercises 7 and 8. 



2 

5 

-3 

-1 


2 

5 

4 

1 


3 

0 

1 

-3 


4 

7 

6 

2 

7. 

-6 

0 

-4 

9 

8. 

6 

-2 

-4 

0 


4 

10 

-4 

-1 


-6 

7 

7 

0 


9. Use detenninant to find out whether the matrix is invertible 


2 

1 

3 

0 


0 

-7 

8 

7 


0 8 
-5 0 
6 0 
5 4 


10. Let A and B be 3x3 matrices, with det A = 4 and det B = -3. Use properties of 
determinants to compute 

(a) det AB (b)det7 A (c) det B r (d) det A r 

(e) det A r A 
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11 Show that 

a l b } a x + hj + c, a x b x c) 

(a) a 2 b 2 a 2 +b 2 +c 2 = a 2 b 2 c 2 

a 2 b 2 a 2 +b 2 + c 3 a 3 b 2 c 3 

a x + b x a, - b x Cj a, hj c, 

(b) a 2 +b 2 a 2 -b 2 c 2 =-la 2 b 2 c 2 

a 2 +b 2 a 3 -b 2 c 3 a 2 b 2 c 3 


12 Show that 



a x + b x t 

a 2 + 

a 2 + b 3 f 


a i 

a., 

a 3 

(a) 

a x t + b x 

a 2 t + b 2 

o 3 f + h 3 

ll 

l 

to 

h 2 

^3 


c i 

C 2 

C 3 


c i 

c 2 

C 3 



a i 

b x + toj 

Cj + rb ] + sa x 


a \ 

a-, 

a 2 

(b) 

a 2 

b 2 + ta 2 

c 2 + rb 2 + sa 2 

= 

b l 

b 2 



a 2 

b 2 + ta 3 

c 3 + rb 2 + sa 2 


c l 

c 2 

C 3 



|i 

2 1 

X X 






13. Show that 1 y y 2 = (y-x)(z-x)(z-y) 

1 2 

l z z 
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Lecture 19 

Cramer’s Rule, Volume, and Linear Transformations 


In this lecture, we shall apply the theory discussed in the last two lectures to obtain 
important theoretical formulae and a geometric interpretation of the detenninant. 

Cramer’s Rule Cramer’s rule is needed in a variety of theoretical calculations. For 
instance, it can be used to study how the solution of Ax = b is affected by changes in the 
entries of b. However, the formula is inefficient for hand calculations, except for 2x2 or 
perhaps 3x3 matrices. 


Theorem 1 (Crammer’s Rule) Let A be an invertible n x n matrix. For any b in R“ 
the unique solution x of Ax = b has entries given by 
det A. (b) 


x. = ■ 


det A 


i = 1,2, 


( 1 ) 


Example 1 Use Cramer’s rule to solve the system 
3xj - 2x 2 = 6 
-5xj + 4 x 2 = 8 

Solution Write the system in matrix form, Ax = b 


1 

<N 

1 

m 

i_ 

i 

5-T 

1_ 


so 

1_ 

i 

1 

_i 

1- 

& 

l_ 


1 

oo 
_1 


where 



" 3 -2l 


X, 


"6" 

A = 

-5 4 

,x = 

I 

_X 2 \ 

&b = 

_8_ 


det ,4 = 


3 

-5 


4(b) = 


-2 

4 

-2 

4 


= 12-10 = 2 
,A 2 (b)= \ 


6 

8 


Since det A = 2, the system has a unique solution. By Cramer’s rule, 


= 


x 2 = 


detyfj(h) _ 24 + 16 
det^4 2 

&etA 2 (b) _ 24 + 30 
det A 2 


= 20 


= 27 
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Example 2 Consider the following system in which 5 is an unspecified parameter. 
Determine the values of s for which the system has a unique solution and use Cramer’s 

3sx { - 2x, = 4 

rule to describe the solution. 

-6x l +sx 2 =1 


Solution Here 



~3s -2] 


~4~ 

A = 

-6 s 

, b = 

1 


4(b) = 


4 

1 


Since detA = 3s 2 -12 = 3(s + 2)(s-2) 

the system has a unique solution when 
det A * 0 

^3(s + 2)(s-2)*0 

=> 4 - 4^0 


^ s *±2 


-2 


s 



4 

1 


For such an 5 , the solution is (xj, X 2 ), where 
det 4 (6) _ 4^ + 2 

1 detH 30 + 2)0-2)’ 

detH 2 0)_ 3s + 24 _ 5 + 8 

* 2_ detH “3(s + 2)(s-2)“(s + 2)(s-2)’ 


sjt± 2 


Example 3 Solve, by Cramer’s Rule, the system of equations 

2xj - x 2 + 3x 3 = 1 

Xj + 2 x 2 - x 3 = 2 
3xj + 2x 2 + 2x 3 = 3 



"2 

-1 

3 ' 


'll 


'1 

-1 

3 " 

Solution Here A = 

1 

2 

-1 

,b = 

2 

> 4 = 

2 

: 2 

-1 


3 

2 

2 



3 


_3 

. 2 

2 


"2 

1 

3 ' 


"2 

-1 

ll 



a 2 = 

1 

2 

-1 

,4 = 

1 

2 : 

2 




3 

3 

2 


3 

2 . 

3 




©Virtual University Of Pakistan 


235 




19-C'ramcr‘s Rule. Volume, and Linear Transformations 


VU 


D = det ^4 = 2 • 6 -1 • (-5) + 3(—4) = 5 


D l = det A x b = 1 • 6 + 2 • 8 + 3(—5) = 7 
D 2 =det^ = 2-(7)-l-(5) + 3(-3) = 0 
a = det 46 = 2 • (2) + 1 • (-3) +1(—4) = -3 


So x. 


A 7 
a 5 2 


^ = 0 , x 3 =^ 

a 3 a 


3 

5 


Example 4 Use Cramer’s Rule to solve. 

Xj + 2 x 3 = 6 

-3xj + 4 x 2 + 6x 3 =30 
- Xj - 2 x 2 + 3x 3 = 8 

Solution 


' 1 

0 

2] 


" 6 " 

-3 

4 

6 

,b = 

30 

-1 -2 

3 


8 


4 = 

" 6 0 2' 

30 4 6 

, 4 — 

' 1 6 2' 

-3 30 6 

4 = 

" 1 0 6" 

-3 4 30 


8-2 3 


1 

1 

OO 

1_ 


1 

1 

1 

K> 

OO 

1 _ 


Therefore, 

_ det(46) _ -40 _ -10 _ det(46) _ 72 _ 18 

X '~ det( A ) ~~4a~~\A' detfid) _ 44 _ TT 

_ det(46) _ 152 _ 38 

~ det(^) “" 44 " ~ IT 

Note For any n x n matrix A and any b in if", let Ai(b) be the matrix obtained from A by 
replacing /th column by the vector b. 

Ai(b) = [a 1 ... b ... a„] 

t 

ith column 

Formula for A~ x 

Cramer’s rule leads easily to a general formula for the inverse of n x n matrix A. The 
jth column of A' 1 is a vector x that satisfies Ax = ej 

where ej is the jth column of the identity matrix, and the ith entry of x is the (i, /)-entry of 
A' 1 . By Cramer’s rule, 

( n det 4(c) 

{(/, j) - entry of A } = x.. = (2) 
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Recall that Ap denotes the submatrix of A formed by deleting row j and column i. A 
cofactor expansion down column i of Ai(ej) shows that 

det A i (e J ) = (-1 T J det A jt = C, f (3) 


where Cp is a cofactor of A. 


By (2), the (z',y')-entry of A' 1 is the cofactor Cp divided by det ,4. 
[Note that the subscripts on Cp are the reverse of (/,/).] Thus 


A~ l =■ 


det A 


C C 

Ml M 

c c 

y-"y 


C,„ C, 


C„ 

c 


c 


(4) 


The matrix of cofactors on the right side of (4) is called the adjugate (or classical 
adjoint) of A, denoted by adj A. (The term adjoint also has another meaning in advance 
texts on linear transformations.) The next theorem simply restates (4). 


Theorem 2 (An Inverse Formula) 

Let A be an invertible n x n matrix, then A 

Example 


- adj A 

detd 


For the matrrix say 
2 3" 


A = 


-1 5 


det ^4 = 10-(-3) = 13 


=> A~‘ will also be a 2x2 matrix 
As 

Aji = submatrix of A formed by deleting row j and column i 
So in this case 

A u = submatrix of A formed by deleting row 1 and column 1 =[5] 
A n = submatrix of A formed by deleting row 1 and column 2 =[-ll 
A 21 = submatrix of A fonned by deleting row 2 and column 1 =[3] 
A 22 = submatrix of A fonned by deleting row 2 and column 2 =[2] 
and 


det4(e.) = (-l)^det(4 ; .) = C.. 
where e j is the jth column of identity matrix I n/n 
So in this case 
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det Afef C, 


C n =det4(e,) = (-l) 1+I det^ n =(+l)det[5] = 5 

C 12 = det ^(e,) = (-1) 1+2 det = (-l)det[-l] = (-1)(-1) = 1 

C 21 = det^(e 2 ) = (-1) 2+1 det^ 21 = (—l)det[3] = -3 

C 22 = det^ 2 (e 2 ) = (-1) 2+2 det A 22 = (+l)det[2] = 2 
By Cramer’s rule, 

{(/, j) - entry of A ' 1 } = x ij = 

So for the current matrix; 

{(1,1)- entry of A~^ = x n = 

{(1,2) - entry of A ~'} = x l2 = 

{(2,1) - entry of A } = x 2l = 

{(2,2) - entry of A~ 1 \=x 22 = 

1 ’ det A 

Hence by using equation # 4, we get 


det A det A 

det4(^)_ C u _ 5 
det A det A 13 

det^l 1 (e 2 )_ C 21 _-3 
~ deC4 ~T3 

1 


det ,4 
detAfef _ C 12 
del A 


det A 13 


del A 1 (e 1 ) C 


22 


detA 13 


A~ = 




" c n 

c 

^21 





“ 5 

-3" 

X 1 j x 12 


det A 

det A 

1 

~c n 

c 

^21 ■ 


13 

13 

X 2l X 22J 


r 

'-'12 

r 

'-'22 

det A 

c l2 

c 

^22 J 


1 

2 



_det,4 

det A_ 





.13 

13. 


Example 5 Find the inverse of the matrix A = 


2 1 
1 -1 
1 4 


3 

1 

-2 


Solution The nine cofactors are 

1-1 


C n = + 


1 


4 -2 


= -2 C = - 
*■•> <^ 12 


1 1 

1 -2 


= 3, C 13 = + 


1 -1 
1 4 


= 5 


r =- 
^21 


1 3 
4 -2 


= 14, C 22 = + 


2 3 

1 -2 


= -7 C =- 


2 1 
1 4 


= -7 


C 31 = + 


1 3 

-1 1 


= 4 C =- 

', '-'32 


2 3 

1 1 


= 1, C 33 = + 


2 1 
1 -1 


= -3 


The adjoint matrix is the transpose of the matrix of cofactors. [For instance, Cn goes in 
the (2, 1) position.] Thus 
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r 

'“'21 

r 

^31 


'-2 

14 

4~ 

adj A = 

C l2 

c 

^22 

c 

'“'32 

= 

3 

-7 

1 


_c 13 

c 

^23 

C 1 

^33 J 


5 

-7 

-3 


We could compute del A directly, but the following computation provides a check on the 
calculations above and produces det.4: 


(i adjA).A = 


"-2 

14 

4 “ 

"2 

1 

3 1 


“14 

0 

0 " 

3 

-7 

1 

1 

-1 

1 

= 

0 

14 

0 

5 

-7 

-3 

1 

4 

-2 


0 

0 

14 


= 14/ 


Since (adj A) A = 14 /, Theorem 2 shows that det^ = 14 and 


“-2 14 

4 “ 


-2/14 

14/14 

4/14 “ 

3 -7 

1 

= 

3/14 

-7/14 

1/14 

5 -7 

-3 


5/14 

-7/14 

-3/14 


Determinants as Area or Volume 

In the next application, we verify the geometric interpretation of detenninants and we 
assume here that the usual Euclidean concepts of length, area, and volume are already 
understood for R 2 and R 3 . 

Theorem 3 If A is a 2x2 matrix, the area of the parallelogram determined by the 
columns of A is |det ^4|. If ,4 is a 3x3 matrix, the volume of the parallelepiped detennined 

by the columns of ,4 is |det^4|. 


Example 6 Calculate the area of the parallelogram detennined by the points (-2, -2), 
(0,3), (4,-1) and (6, 4). 


Solution 

Let A(-2,-2), B(0,3), C(4,-l) and D(6,4). Fixing one point say A(-2,-2) and find the 
adjacent lengths of parallelogram which are given by the column vectors as follows; 


AB = 


0 - (- 2 ) 
3 -(- 2 ) 


2 

5 


AC = 


1 

CnT 

1 

1_ 


" 6 " 

_—1 (—2) J 


1 


So the area of parallelogram ABCD determined by above column vectors 



6 

1 


2-30 = -28 =28 


Now we translate the parallelogram ABCD to one having the origin as a vertex. For 
which we subtract the vertex (-2, -2) from each of the four vertices. The new 
parallelogram has the vertices say 
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Solution We claim that E is the image of the unit disk D under the linear transformation 

a 0 


A:D—>E detennined by the matrix A = 


Au = x where u = 


e D , x = 


0 b 
eE. 


, given as 


a 

0" 

u \ 


Xi 

0 

b 

u 2 


_x 2 _ 


Now Au = x 


an. 


bu. 


then 


au ] = Xj and bu 2 = x 2 


x, . x, 

u, = —- and u, = — 
a b 

Since u e D (in the circular disk),it follows that the distance of u from origin will be less 
than unity i-e 

(if- -0) + (w2 — o) <1 



Hence by the generalization of theorem 4, 

{area of ellipse} = {area of A(D)\ (here T = A) 

= |det^4|. {area ofD} 

= ab.n{ 1) = ^ab 


U2 



X2 1 
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Example 8 Let S be the parallelogram determined by the vectors b t = ^ and b 2 = ^ 
fi -ll 

and let A = ^ ^ . Compute the area of image of S under the mapping x —> Ax . 

= 14 , and det^ = 2. By theorem 4, the area of 
image of S under the mapping x —> Ax is |det^4|. {area of S} = 2.14 = 28 


Solution The area of S is det 
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Exercises 


Use Cramer’s Rule to compute the solutions of the systems in exercises 1 and 2. 


2xj + x 2 =7 


2xj +x 2 + x 3 = 4 


1 . -3xj + 


x 2 + 2x 3 


-8 

-3 


2 . -Xj + 


2x, 


3xj +x 2 + 3 x 3 


In exercises 3-6, determine the values of the parameter s for which the system has a 
unique solution, and describe the solution. 


6sx, + 4 x 2 = 5 
9xj + 2sx, = -2 


3sx, -5x 0 = 3 
4. 

9xj + 5sx , = 2 


sx, - 2sx, = -1 
5. ; 2 

3x, + 6sx 2 = 4 


2sx, + x, = 1 

6 . 1 

3sx 2 + 6sx 2 = 2 


In exercises 7 and 8, compute the adjoint of the given matrix, and then find the inverse of 
the matrix. 


"3 

5 

4] 


"3 

0 

0 " 

1 

0 

1 

8. 

-1 

1 

0 

2 

1 

1 


-2 

3 

2 


In exercises 9 and 10, find the area of the parallelogram whose vertices are listed. 

9. (0, 0), (5, 2), (6, 4), (11, 6) 10. (-1, 0), (0, 5), (1, -4), (2, 1) 

11. Find the volume of the parallelepiped with one vertex at the origin and adjacent 
vertices at (1, 0, -2), (1, 2, 4), (7, 1, 0). 

12. Find the volume of the parallelepiped with one vertex at the origin and adjacent 
vertices at (1, 4, 0), (-2, -5, 2), (-1, 2, -1). 


13. Let S be the parallelogram detennined by the vectors bi = 


-2 

3 


and b 2 = 


-2 

5 


and 


let ,4 = 


6 -2 

-3 2 


. Compute the area of the image of S under the mapping x —> Ax . 
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14. Let S be the parallelogram determined by the vectors bi = 


4 

-7 


and b 2 = 


0 

1 


, and 


let ^4 = 


7 

1 


2 

1 


Compute the area of the image of S under the mapping x —> Ax . 


15. Let T: R 3 —» R 3 be the linear transfonnation determined by the matrix 
a 0 0 | 

where a, b, c are positive numbers. Let S be the unit ball, whose 


0 6 0 
0 0c 
bounding surface has the 


equations/ + x/ + xf = 1. 


x , , *2 


a. Show that T (, S) is bounded by the ellipsoid with the equation —— + 

a b 


2 2 

X 3 r 
+ ^T = 1. 

C~ 


b. Use the fact that the volume of the unit ball is 4n / 3 to detennine the volume of the 
region bounded by the ellipsoid in part (a). 
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Lecture 20 

Vector Spaces and Subspaces 


Case Example 

The space shuttle's control systems are absolutely critical for flight. Because the 
shuttle is an unstable airframe, it requires constant computer monitoring during 
atmospheric flight. The flight control system sends a stream of commands to 
aerodynamic control surfaces. 

Mathematically, the input and output signals to an engineering system are functions. It 
is important in applications that these functions can be added, and multiplied by 
scalars. These two operations on functions have algebraic properties that are completely 
analogous to the operation of adding vectors in R " and multiplying a vector by a scalar, 
as we shall see in the lectures 20 and 27. For this reason, the set of all possible inputs 
(functions) is called a vector space. The mathematical foundation for systems 
engineering rests on vector spaces of functions, and we need to extend the theory of 
vectors in R" to include such functions. Later on, we will see how other vector spaces 
arise in engineering, physics, and statistics. 

Definition Let V be an arbitrary nonempty set of objects on which two operations are 
defined, addition and multiplication by scalars (numbers). If the following axioms are 
satisfied by all objects u, v, w in V and all scalars k and /, then we call V a vector space. 

Axioms of Vector Space 

1. Closure Property For any two vectors u & v e V, implies u + v e V 

2. Commutative Property For any two vectors «&vef implies u + v = v + u 

3. Associative Property For any three vectors u, v, w e V, u + (v + w) = (u + v) + w 

4. Additive Identity For any vector u e V, there exist a zero vector 0 such that 

0+u=u+0=u 


5. Additive Inverse For each vector u e V, there exist a vector u in V such that 

-u + u = 0 = u + (-u) 

6. Scalar Multiplication For any scalar k and a vector u e V implies k u e V 

7. Distributive Law For any scalar k if u & v eV, then k (u + v) = k u + kv 

8. For scalars m, n and for any vector u e L, (m + n) u = m u + nu 

9. For scalars m, n and for any vector u e V, m (n u) = (m n) u = n (m u) 
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10. For any vector u eV, lu = u where 1 is the multiplicative identity of real numbers. 

Examples of vector spaces The following examples will specify a non empty set V and 
two operations: addition and scalar multiplication; then we shall verify that the ten vector 
space axioms are satisfied. 

Example 1 Show that the set of all ordered n-tuple R" is a vector space under the 
standard operations of addition and scalar multiplication. 

Solution 

(i) Closure Property: 

Suppose that u = (ui, U 2 , .... u n ) and v = (vi, vi, .... v„) e R" 


Then by definition, u + v = (uj, U 2 , .... u„) + (vi, v?, ..., v„) 

= (ui + vi, U 2 + V 2 , .... u n + v n ) e R" (By closure property) 

Therefore, R" is closed under addition. 

(ii) Commutative Property 

Suppose that u = (ui, U 2 , .... u„) and v = (vi, v?. .... v n ) e R" 

Now U + V = (ui, U 2 , .... u n ) + (vj, V 2 , .... v„) 

= (ui + vi, U 2 + V 2 , .... u n + v n ) (By closure property) 

= (vi + ui, V 2 + U 2 , .... v„ + u n ) (By commutative law of real numbers) 

= (vi, v 2 , .... v n ) + (ui, U 2 , .... u n ) (By closure property) 

= V + u 

Therefore, R" is commutative under addition. 

(iii) Associative Property 

Suppose that u = (ui, U 2 , .... u n ), v = (vi, V 2 , .... v„) and w = (wi, w?, ..., w,,)eR" 

Now (u + v) + W = [(ui, U 2 , .... u n ) + (V/, V 2 , .... V„)] + (wi, W 2 , .... W„) 

= (ui + vi, U 2 + V 2 , .... u n + v„) + (wi, m> 2 , ■■■, w n ) (By closure property) 

= ((ui + vi) + wi, (U 2 + v 2 ) + W 2 , .... (u n + v n ) + w,,)) (By closure property) 

= (ui + (vi + wi), U 2 + (v 2 + W 2 ), .... u n + (v n + w n )) (By associative law of real numbers) 
= (ui, u 2 , .... u n ) + (vi + wi, v 2 + W 2 , .... v„ + w„) (By closure property) 


©Virtual University Of Pakistan 


246 




20-Vector Spaces and Subspaces 


VU 


= (ui, U 2 , u„) + [(vi, v 2 , .... v n ) + (wi, w 2 , .... w„)] (By closure property) 

= u + (v + w) 

Hence R" is associative under addition. 

(iv) Additive Identity 

Suppose u = (ui, ui, ..., u n ) e R". There exists 0 = (0, 0, ..., 0) e R" such that 
0 + U = (0, 0, ..., 0) + (ill, U2, .... Un) 

= (0 + ui, 0 + U 2 , .... 0 + u n ) (By closure property) 

= (ai, U 2 , ..., u n ) = u (Existence of identity of real numbers) 

Similarly, u + 0 = u 

Hence 0 = (0, 0, ..., 0) is the additive identity for R". 

(v) Additive Inverse 

Suppose u = (ui, U 2 , ..., ii n ) g R". There exists -u = (-ui, - 112 , ..., -u n )^R" 

Such that u + (-u) = (ui, 112 , ..., u n ) + (-ui, -U 2 , ..., -u n ) 

= (uj + (-Ui), u 2 + (-U 2 ), ..., u n + (-u n )) (By closure property) 

= (0, 0, ..., 0)=0 
Similarly, (-u) + u = 0 

Hence the inverse of each element of R" exists in R". 

(vi) Scalar Multiplication 

If k is any scalar and « = (u 1 , U 2 , .... u„) e R". 

Then by definition, k u = k (ui, 112 , .... u n ) = (k u 1 , k 112 , ..., k u n ) e R" 

(By closure property) 

(vii) Distributive Law 

Suppose k is any scalar and u = (ui, u 2 , ..., u n ), v = (vi, v 2 , ..., v n ) e R" 

Now k (u + v) = k [(it 1 , ll 2 , ..., u n ) + (vi, V 2 , .... v n )] 
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= k (ui + VI, U 2 + V 2 , Un + v n ) 

= (k (ui + Vi), k (112 + V 2 ), .... k (u n + v n )) 

= (k ui + kvi, k U 2 + kv 2 , k u n + k v n ) 

= (kill, kii 2 , .... ku n ) + (kvi, kv 2 , .... kv n ) 

= k (ill, U2, ..., Un) + k (Vl, V2, ..., Vn) 

= ku + kv 

(viii) Suppose k and / be any scalars and u = (ui. 
Then (k + l) u = (k + l) (ui, U 2 , .... u„) 

= ((k + 1)U1, (k + l)U 2 , ..., (k + l)u„) 

= (kui + l UI, k 112 + l U2, ..., k U„ + I lln) 

= (k U1, k ll2, ..., k Un) + 0 u 1, 1 U2, ..., / Un) 

= k (ui, U2, ..., Un) + l (ill, U2, Un) 

= ku + /u 

(ix) Suppose k and / be any scalars and u = 

Then k(lu) = k [l (ui, U 2 , .... u n )] 

= k (l U], / U2, .... / ii n ) 

= (k (l ui), k (l U 2 ), k (l u n )) 

= ((kl)ui, (k l)u 2 , ..., (kl)u n ) 

= (k l) (u 1 , u 2 , ..., u n ) 

= (kl) u 

(x) Suppose u = (ui, u 2 , ..., u n ) g R" 

Then 1 u = 1 (ui, 112 , u n ) 

= (111 1 , IU 2 , .... lu n ) 


(By closure property) 

(By scalar multiplication) 

(By Distributive Law) 

(By closure property) 

(By scalar multiplication) 

112 , u n ) g R" 

(By scalar multiplication) 

(By Distributive Law) 

(By closure property) 

(By scalar multiplication) 

2 , u n ) G R" 

(By scalar multiplication) 
(By scalar multiplication) 

(By associative law) 

(By scalar multiplication) 


(By scalar multiplication) 


©Virtual University Of Pakistan 


248 




20-Vector Spaces and Subspaces 


VU 


= (u/, u 2 , u n ) = u (Existence of identity in scalrs) 

Hence, R" is the real vector space with the standard operations of addition and scalar 
multiplication. 

Note The three most important special cases of R" are R (the real numbers), R 2 (the vectors 
in the plane), and R 3 (the vectors in 3-space). 

Example 2 Show that the set V of all 2x2 matrices with real entries is a vector space if 
vector addition is defined to be matrix addition and vector scalar multiplication is defined to 
be matrix scalar multiplication. 


Solution Suppose that u = 

U 11 U 12 

,v = 

V l, V 12 

and w = 

w n w 12 


R 

Ki 

N> 


_ V 21 V 22 _ 


<N 

"-i 

£ 


and k and / be two any scalars. 


(i) Closure property To prove axiom (i), we must show that u + v is an object in V: that is , 
we must show that u + v is a 2x2 matrix. But this is clear from the definition of matrix 


addition, since u + v = 

u n 

U 12 

+ 

V ,1 

V 12 

— 

Un+Vn 

U 12 + V 12 


_ U 21 

U 22 _ 


_ V 21 

V„_ 


_ U 21 + V 21 

R 

+ 

Ni 

1_ 


(By closure property) 

(ii) Commutative property Now it is very easy to verify the Axiom (ii) 


U + V — 

U 11 

U 12 

+ 

v ,i 

V 12 

— 

Uu+Vn 

U 12 + V 12 


_ U 21 

R 

Kj 

1 _ 


y?i 

< 

K> 

K) 

1 _ 


_ U 21 + V 21 

R 

Kj 

+ 

1_ 


v n + U 11 V 12 + U 12 


V 2J + U 2l V 22 + U 22 


(By closure property) 
(Commutative property of real numbers) 


V 11 V 12 
V 21 V 22. 


U 11 U 12 
U 21 11 22 


V + U 


/ 

(iii) Associative property (u + v) + w = 

u n 

11 /2 

+ 

V 11 

V 12 

\ 

+ 

Wn 

W n 

V 

_ U 21 

U 22 _ 


y 2i 

V 22 _ 

) 

W 2I 

W 22_ 


Un+Vn 

U 12 + V 12 

+ 

w,i 

W ]2 

U 21 + V 21 

U 22 V 22 _ 


1 

1 


(u n +V n ) + W n (u 12 +v 12 ) + w 12 
(u 2 i + v 2 i) + w 2 i (u 22 + V 22 ) + w 22 

U n +(v n + w n ) U 12 +(v 12 +w 12 ) 

U 21 + (V 2 1 +W 2l) U 22 + ( v 22 + ) 


(By closure property) 


(By associative property of real numbers) 
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u n 

U 12 

+ 

v„+w n 

V ]2 +Wi 2 



U 21 

U 22_ 


V 21 +W 21 

V 22 +W 22_ 



U ll 

U 12 

+ 

' V 11 V 12 

+ 

1 - 

<N 

1_ 

\ 

= u + (v + rv ) 

U 21 

U 22 _ 


V 21 V 22_ 


1 

K) 

1_ 

/ 


Therefore, V is associative under ‘+’. 

(iv) Additive Identity Now to prove the axiom (iv), we must find an object 0 in V such 


that 0 + v = v + 0 = v for all u in V. This can be done by defining 0 — 


0 

0 


0 

0 


0 + u = 

0 

0 

+ 

Un ll 12 


0 

0 


u 2] u 22 ~ 


0 + Ujj 

0 + U n 


u n 

U ]2 

_0 + u 2 i 

0 + u 22 _ 


_ U 21 

1 

<N 

s 


and similarly u + 0 = u. 

(v) Additive Inverse Now to prove the axiom (v) we must show that each object u in V 
has a negative -u such that u + (-u) = 0 = (-n) + 0. Defining the negative of u to be 


-u — 


-u 


11 


-u 


21 


-u 

-u 


12 

22 


u + (-u) = 


U]1 u 12 


U 21 11 22 


~ U U ~ U 12 


U 21 ~ U 22 


*11 


*21 


+ u 12 +( u l2 ) 

( ~ U 21 ) U 22 + ( ~ U 22 ) 


0 0 
0 0 


0 


Similarly, (-«) + u = 0 

(vi) Scalar Multiplication 

Axiom (vi) also holds because for any real number k we have 


ku = k 


1 

_1 


1 

r 

r 
_1 

l 

R 

N» 

R 

K> 

Ni 

1_ 


_ku 2] 

1 

-Si 


(By closure property) 


so that k u is a 2x2 matrix and consequently is an object in V. 

(vii) Distributive Law 

k ( u + v ) = k 


f 

1 

11 12 

+ 

1 



V 

_ U 21 

U 22_ 


1 

< 

N) 

V 22_ 

J 
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= k 


u n +v n 


U 21 +V 21 


11 12 + V 12 


U-22 ' ^22 


k(u n +v n ) k(u 12 +v 12 ) 
k(u 21 +v 21 ) k(u 22 +v 22 ) 


1 

r 

+ 

5* 

r 

+ 

_1 


1 

r 

r 

_1 

+ 

kVn 

kv n 

L ku 21 +kv 2J 

ku 22 + kv 22 _ 


\ku 2 i 

ku 22 _ 


_kv 2i 

kV 22_ 


k 

u n 

U] 2 

+ k 

v n 

V 12 


_ U 21 

11 22 _ 


_ V 21 

V 22 _ 


ku + kv 


(k + l)u n (k + l)u 12 
(k + l)u 21 (k + l)u 12 


ku n + lu n ku 12 + lu 12 
ku 2I + lu 2I ku 22 + lu 72 


_ 

1 

r 

ku, 2 

+ 

1 

1 

'--o* 


\ku 2 i 

ku 2 2 


\lu 2 i 

R 

K> 

1_ 


(viii) (k + l)u — (k + l) 


U,1 U 12 


U 2 1 u 22 


k 

1 

J2 

U 12 

+ / 

1 

s 

1 

sT 


L U 21 

U 22 _ 


1 

R 

K) 

i 

<N 

R 


ku + lu 


(ix) k(lu) = k 


f 

l 

U/I 

U 12 

\ 

= k 

lu u 

lu n 

\ 

_ U 21 

U 22 _ 

) 

_lu 2 i 

lu 22 _ 


k(lu n ) k(lu 12 ) 
k(lu v ) k(lu 22 ) 


(kl)u n 

(kl)u 12 

II 

1 

^5 

1 

<N 

r" 

(kl)u v 

K 

Kj 

K> 

1_ 

<N 

R 

_1 

i 

<N 

<N 

r 


(kl)u 


(x) Finally axiom (x) is a simple computation 


U 11 

Ki 

_ 1 


1 

^5 

_ 1 


U 11 U 12 

U 21 

R 

Kj 

1 _ 


J U 21 

R 

Kj 

1 _ 


u 2 i u 22 


Hence the set of all 2x2 matrices with real entries is vector space under matrix addition 
and matrix scalar multiplication. 

Note Example 2 is a special case of a more general class of vector spaces. The arguments in 
that example can be adapted to show that a set V of all mxn matrices with real entries, 
together with the operations of matrix addition and scalar multiplication, is a vector space. 
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Example 3 Let Fbe the set of all real-valued functions defined on the entire real 
line (- 00 , oo). If fge V, then f+g is a function defined by 

(f +g) (x) =f(x) + g (x), for all xeR. 

The product of a scalar aeR and a function/in Kis defined by 
(af)(x)=af (x), for all xeR. 

Solution 

(i) Closure Property If/, ge V, then by definition 

(f+g) (x) =f(x) + g (x) e V. Therefore, V is closed under addition. 

(ii) Commutative Property If/ and g are in V, then for all xeR 

(f+g) (x) =f(x) +g (x) (By definition) 

= g (x) + f (x) (By commutative property) 

= (g + f) (x) (By definition) 

So that f + g = g+f 

(iii) Associative Property If/, g and h are in V, then for all xeR 

((f+g) + h) (x) = (f+ g) (x) + h (x) (By definition) 


= (f(x)+g (x)) + h (x) 
= f(x) + (g (x) + h (x)) 
=f(x) + (g + h) (x) 


(By definition) 

(By associative property) 
(By definition) 


= (f+(g + h))(x) 

And so (f+g) + h= f+(g + h) 

(iv) Additive Identity The additive identity of V is the zero function defined by 

0 (x) = 0 , for all xeR because (0 + f) (x) = 0 (x) + f (x) (By definition) 

= 0 +f (x) = f (x) (Existence of identity) 

i.e. 0 +/=/ Similarly,/+ 0 =f 

(v) Additive Inverse The additive inverse of a function/in Kis (-1) /= -fe Kbecause 

(f + (-f)) (x) =f(x) + (-f) (x) (By definition) 

=/ (x) -f (x) (By definition) 

= 0 (Existence of inverse) 
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i-e./+ (-J) = 0. Similarly, (-J) +f = 0. 

(vi) Scalar Multiplication If/is in V and a is in R, then by definition (a f) (x) = a f (x) 
e V. 

(vii) Distributive Law If f g are in V and aeR, then 

(a(f+ g)> (x) = a (f+g) (x)=a(f (x) + g (x)) = af(x) + a g (x) 

= (a f) (x) + (a g) (x) = (a f + a g) (x) and, therefore, a (f + g) = a f + a g 
(viii) Let a, b in R and /e V, then 

((a + b)f) (x) = (a + b)f(x) = af(x) + bf(x) = (af) (x) + (bf) (x) = (af+ bf) (x) 
Thus (a + b)f= af + bf 

(ix) a (bf) (x) = a (bf (x)) = (a b) f (x) showing that a (bf) = (ab)f 

(x) (l.f) (x) = 1 f (x) =f (x) (Existence of identity) 

And so /./ = / 

Hence V is a real vector space. 

Example 4 If p(x) = a 0 + a : x + a 7 x 2 + ... + a n x" 

and q(x) = b 0 + b,x + b 2 x 2 + ... + b n x" 

We define 

p(x) + q(x) = (a 0 + cijX + a 2 x 2 +... + a n x") +(b 0 + b,x + b 2 x 2 +... + b n x n ) 

= (a n +b 0 ) + (cij +bj)x + (a 2 +b 2 )x 2 + ... + (a n +b n )x" and for any scalar k, 

kp(x)= k(a 0 + a,x + a 2 x 2 +... + a n x n ) = ka 0 + ka,x + ka^x 2 + ... + k a n x" 

Clearly the given polynomial is a vector space under the addition and scalar 
multiplication. 

Example 5 (The Zero Vector Space) Let V consists of a single object, which we define 
by 0 and 0 + 0=0 and k 0 = 0 for all scalars k. It is easy to check that all the vector space 
axioms are satisfied. We call V={0} as the zero vector space. 
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Example 6 (Every plane through the origin is a vector space) 

Let Kbe any plane through the origin in R 3 We shall show that the points in V form a 
vector space under a standard addition and scalar multiplication operations for vectors in 
R 3 . 

From example 1, we know that R 3 itself is a vector space under these operations. Thus, 
Axioms 2, 3, 7, 8, 9 and 10 hold for all points in R 3 and consequently for all points in the 
plane V. We therefore need only show that Axioms 1, 4, 5 and 6 are satisfied. 

Since the plane is passing through the origin, it has an equation of the form 

ax + by + cz = 0 ( 1 ) 

Thus, if« = (ui, U 2 , U 3 ) and v = (v 1 , vo, vj) are points in V, then 

a ui + b U2 + c U3 = 0 and a vi + b V 2 + c V 3 = 0. 

Adding these equations gives a (ui + v{) + b (112 + V2) + c (113 + V3) = 0 

This equality tell us that the coordinates of the point 


U + V = (ill + v 1 , Il 2 + V 2 , U 3 + Vs) 


satisfies (1); thus, u + v lies in plane V. This proves that the Axiom 1 is satisfied. 

There exists 0 = (0, 0, 0) such that a (0) +b (0) + c (0) = 0. Therefore, Axiom 4 is 
satisfied. 

Multiplying a ui + b 112 + c u .3 = 0 through by k gives 
a (kui) + b (kii 2 ) + c (kus) = 0 

Thus, (ku 1 , k u 2 , kus) = k (u 1 , 112 , 113 ) = kue V. Hence, Axiom 6 is satisfied. 

We shall prove the axiom 5 is satisfied. Multiplying a ui + b u2 + c 113 = 0 through by -1 
gives a (-lui) + b (-hi 2 ) + c (-lu 3 ) = 0 

Thus, (-ui, - 1 * 2 , - U 3 ) = - (ui, U 2 , U 3 ) = -hgV. This establishes Axiom 5. 

Example 7 (A set that is not a vector space) 

Let V=R 2 and define addition and scalar multiplication operation as follows. If 
u = (u 1 ,u 2 )and v = (v l ,v 2 ) then define 

u + v = (Uj +Vj,u 2 +v 2 ) and if k is any real number then define ku = (kUj,0). 

For any vector u eV, lu =l(ui, 112 ) = (1 ui, 0) = (u /, 0) ^ u where 1 is the multiplicative 
identity of real numbers. Therefore, the axiom 10 is not satisfied. 

Hence, V=R 2 is not a vector space. 
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Theorem 1 Let Fbe a vector space, u a vector in V, and k is a scalar, then 
(i) Ou = 0 (ii) kO = 0 


(iii) (-1) u = -u (iv) If k u = 0 then k = 0 or u = 0 

Definition A subset W of a vector space V is called a subspace of V if W itself a vector 
space under the addition and scalar multiplication defined on V. 

Note If IF is a part of a larger set V that is already known to be a vector space, then 
certain axioms need not be verified for IF because they are “inherited” from V. For 
example, there is no need to check that u + v = v + u (Axiom 2) for IF because this holds 
for all vectors in V and consequently for all vectors in IF. Other Axioms are inherited by 
IF from V are 3, 7, 8, 9, and 10. Thus, to show that a set IF is a subspace of a vector space 
V, we need only verify Axioms 1, 4, 5 and 6. The following theorem shows that even 
Axioms 4 and 5 can be omitted. 

Theorem 2 If IF is a set of one or more vectors from a vector space V, then IF is 
subspace of V if and only if the following conditions hold. 

(a) If u and v are vectors in IF, then u + v is in IF 

(b) If k is any scalar and u is any vector in IF, then k u is in IF. 

Proof If IF is a subspace of V, then all the vector space axioms are satisfied; in 
particular, Axioms 1 and 6 hold. But these are precisely conditions (a) and (b). 

Conversely, assume conditions (a) and (b) hold. Since these conditions are vector space 
Axioms 1 and 6, we need only show that IF satisfies the remaining 8 axioms. The vectors 
in IF automatically satisfy axioms 2, 3, 7, 8, 9, and 10 since they are satisfied by all 
vectors in V. Therefore, to complete the proof, we need only to verify that vectors in IF 
satisfy axioms 4 and 5. 

Let u be any vector in IF. By condition (b), k u is in IF for every scalar k. Setting 
k = 0, it follows from theorem 1 that 0 u = 0 is in IF, and setting k = - 1, it follows that 
(-1) u = -U is in IF. □ 


Remark 

(1) The theorem states that IF is a subspace of V if and only if IF is closed under addition 
and closed under scalar multiplication. 

(2) Every vector space has at least two subspaces, itself and the subspace {0} consisting 
only of the zero vector. Thus the subspace {0} is called the zero subspace. 

Example 8 Let IF be the subset of R 3 consisting of the all the vectors of the form 
(a, b, 0), where a and b are real numbers. To check if IF is subspace of R 3 , we first see 
that axiom 1 and 6 of a vector space holds. 
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Let u = (a 1 ,b 1 ,0) and v — (a,,b 2 ,0) be vectors in W then 

u + v = (aj.bj.O) J r(a 2 ,b 1 ,0) = (a 1 + a 2 ,b 1 +b 2 ,0) is in W. Since the third component is 
zero. Also c is scalar, and then cu = c(a,,b,,0) = (ca lt cb lt 0) is in W. Therefore the I st and 

6 th axioms of the vector space holds. We can also verify the other axioms of vector space. 
Hence W is a subspace. 


Example 9 Consider the set W consisting of all 2x3 matrices of the fonn 
a b 0 


0 c d 
subspace M 2x $- 


Where a, b, c and d are arbitrary real numbers. Show that the W is a 



a, b, 0 


a, b , 0 

Solution Consider u = 

1 1 

,V = 

Z Z 


i 

o 

n 

i_ 


1 

<N 

<N 

o 

_1 


in W 


Then u + v = 

i 

0" 

+ 

a 2 

b 2 

0" 

— 

a, + a 2 

b,+b 2 

0 


i 

o 

d, 


0 

C 2 

1_ 


0 

c,+c 2 

d, + d 2 


is in W. 


So that the (a) part of the theorem is satisfied. Also k is a scalar, and then 
ka j kb, 0 

ku — is in W. So the (b) part of the above theorem is also satisfied. 

0 kc, kd, J 

Hence W is a subspace of M 2x2 . 


Note Let Kis a vector space then every subset of V is not necessary a subspace of V. 
For example, let V=R 2 then any line in if 2 not passing through origin is not a subspace of 

i 2 . ... ? 

if'. Similarly, a plane in if not passing through the origin is not a subspace of if . 

Example 10 Let Wbe the subset of if 3 consisting of all vectors of the form (a, b, 1), 
where a, b are any real numbers. To check whether property (a) and (b) of the above 
theorem holds. Let u = (a,,b,,l)and v = (a 2 ,b 2 ,l)be vectors in W. 

Then u + v = (a I ,b 1 ,l) + (a 2 ,b 2 ,l) = (a, + a 2 ,b, + b 2 ,l +1) which is not in JEbecause 
the third component 2 is not 1. As the 1st property does not hold therefore, the given set 
of vectors is not a vector space. 

Example 11 Which of the following are subspaces of if 3 

(i) All vectors of the form (a, 0, 0) 

(ii) All vectors of the form (a, 1,1) 

(iii) All vectors of the form (a, b, c), where b = a + c 

(iv) All vectors of the form (a, b, c), where b = a + c +1 


Solution Let W is the set of all vectors of the fonn (a, 0, 0). 


(i) Suppose u = (ui, 0, 0) and v = (vi, 0, 0) are in W. 

Then u + v = (u j, 0, 0) + (vj, 0, 0) = (uj + vj, 0, 0) which is of the form (a, 0, 0). 


©Virtual University Of Pakistan 


256 




20-Vector Spaces and Subspaces 


VU 


Therefore, u + veW 

If k is any scalar and u = (ui, 0, 0) is any vector in W, then ku = k (m, 0, 0) = (k u /, 0, 
0 ) 

which is of the form (a, 0, 0). Therefore, k u e W. Hence W is the subspace of R 3 . 


(ii) Let W is the set of all vectors of the form (a, 1,1). 

Suppose u = (ui, 1, 1) and v = (V/, 1, 1) are in W. Then u + v = (ui, 1, 1) + (vj, 

1, 1) = (ui + vj, 2 , 2 ) which is not of the form (a, 1, 1). Therefore, u + v<£W. Hence W is 
not the subspace of R 3 . 

(iii) Suppose W is the set of all vectors of the form (a, b, c), where b = a + c 
Suppose u = (ui, ui + u 3 , U 3 ) and v = (vi,vi + V 3 , V 3 ) are in W. 


Then u + v = (uj, ui + u3, 113) + (vi, v; + v3, V3) 

= (ui + Vi, Ui + 113 + Vl + V3, 113 + V3) 

= (ui + vi, (ui + vi) + (U 3 + V3), 113 + V3), which is of the form (a, a + c, c). 
Therefore, u + veW 

If k is any scalar and u = (u 1, ui + u3, u3) is any vector in W, then 

ku = k (ui, ui + U3, 113) = (kui, k (ui + U3), kus) (By definition) 

= (k ui, k ui + k U 3 , k U 3 ) (By Distributive Law) 

Which is of the form (a, a + c, c). Therefore, k u e W. Hence W is the subspace of R 3 . 

(iv) Let W is the set of all vectors of the form (a, b, c), where b = a + c +1 
Suppose u = (ui, ui + u 3 + 1, u 3 ) and v = (vi, vi + V 3 + 1, V 3 ) are in W. 

Then u + v = (ui, ui + u 3 + 1, u 3 ) + (vi, vi + v 3 + 1, v 3 ) 

= (ill + Vi, U] + U3 + 1 + V1 + V3 + 1 , U3 + V3) 

= (u 1 + v 1, (ui + Vi) + (113 + v3) + 2 , 113 + v3) 

Which is not of the form (a, a + c + 1, c). Therefore, u + vg W. Hence W is not the 
subspace of R 3 . 

Example 12 Determine which of the following are subsnaces of P?. 

(i) All polynomials ao + ai x + 02 x +a 3 ;c for which ao = 0 

(ii) All polynomials ao + ai x + a 2 x 2 + a 3 x 3 for which ao + a 1 + a2 + as = 0 

(iii) All polynomials ao + ai x + a2 x + a3 x 3 for which ao, a 1, as, and a3 are integers 

(iv) All polynomials of the form a 0 + a / x, where a 0 and a 1 are real numbers. 
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2 5 

Solution (i) Let W is the set of all polynomials ao + ai x + a 2 x + a 3 x for which an = 
0. 

Suppose that u = cq + cj x + C 2 x 2 + C 3 x 3 (where co = 0) and v = bo + bi x + b 2 x 2 + bs 
x (where bo = 0) are in W. Then u + v = (co + cj x + C 2 X + c 3 x ) + (bo + bix + b 2 x + 

3 2 3 

bsx) = (co + bo) + (ci + bj)x + (C 2 + b 2 )x + (c 3 + bs) x , where co + bo = 0 . 
Therefore, u + v eW 

2 3 

If k is any scalar and u = co + cj x + C 2 x + C 3 x (where co = 0) is any vector in W. 
Then k u = k (co + cix + C 2 X + c 3 X ) = (kco) + (kci) x + (kc 2 ) x + (kc 3 ) x where 
k co = 0. Therefore, k ueW. Hence W is the subspace of P 3 . 

(ii) Let W is the set of all polynomials ao + aix + a 2 X + a 3 X for which 

ao+a i+a 2 +a 3 = 0 . 

Suppose that u = co + cj x + C 2 x 2 + C 3 X 3 (where co + ci + c? + C 3 = 0) and 

2 3 

v = bo + bi x + bs x + b3 x (where bo + bi + b 2 + b 3 = 0) are in W. 

Now 

u + v = (c 0 + ci x + C 2 x + C 3 x ) + (bo + bj x + b 2 x +b 3 x) 

= (Co + bo) + (c ! + b,) x + (c 2 + b 2 ) x 2 + (c 3 + b 3 ) x 3 

Where (co+bn) + (ci+bi) + (c 2 +b 2 ) + (cs+bs) = (C 0 +C 1 +C 2 +C 3 ) + (bo+bi+bs+bs) = 0 
+ 0 = 0. Therefore, u + v eW 

If k is any scalar and u = co + cj x + c 2 x + C 3 x (where co + cj + c 2 + C 3 = 0) is any 
vector in W. Then k u = k (cq + cix + c 2 x 2 + C3X 3 ) = (kco) + (kci) x + (kc 2 ) x 2 + (kcs) 
x 3 

Where (kco) + (kci) + (kc 2 ) + (k C3) = k(co + cj + c 2 + C3) = k.O = 0 
Therefore, kueW. Hence W is the subspace of P3. 

2 3 

(iii) Let W is the set of all polynomials ao + ai x + a 2 x + a 3 x for which ao, a 1 , a 2 , 
and a 3 are integers. 

Suppose that the vectors u = co + ci x + c 2 x + c 3 x (where co, c 1 , c 2 , and c 3 are 
integers) and v = bo + bj x + b 2 x 2 + bs x 3 (where b 0 , bj, b 2 , and b 3 are integers) are in 
W. 

Now 

u + v = (co + c 1 x + c 2 x 2 + C 3 x 3 ) + (bo + bi x + b 2 x 2 + b 3 x 3 ) 

= (co + b 0 ) + (cj + bi) x + (c 2 + b 2 ) x + (c 3 + b 3 ) x , where 

(co + bo), (ci + bj), (c 2 + b 2 ), and (c 2 + b 3 ) are integers (integers are closed under 

addition). Therefore, u + v eW 

If k is any scalar and u = Co + cj x + c 2 x + C 3 x (where co, c 1 , c 2 , and c 3 are integers) 

is any vector in W. Then ku = k (co + cix + c 2 x + c 2 x = (kco) + (kci) x + (kc 2 ) x + 

(kc 3) x 3 , where (kco), (kcj), (k c 2 ), and (kcs) are not integers (product of real number 
and integer). Therefore, ku<£W. Hence, W is not the subspace of P 3 . 
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(iv) Let W is the set of all polynomials of the form cio + a i x, where ao and a / are real 
numbers. Suppose that u = co + ci x (where Co and ci are real numbers) and 
v = bo + bj x (where bo and bi are real numbers) are in W. 

Then u + v = (co + cj x) + (bo + bj x) = (co + bo) + (ci + bj) x 
Where (co + b 0 ) and (a + bi) are real numbers. 

Therefore, u + v eW 


If k is any scalar and u = co + ci x (where Co and c 1 are real numbers) is any vector in W. 
Then k u = k (co + cj x) = (k co) + (kci)x 
Where (kco) and (kci) are real numbers. 

Therefore, kueW. Hence W is the subspace of P 3. 

Example 13 Determine which of the following are subspaces of M 22 . 
a b 

where a + b+c + d = 0 


(i) All matrices 


c d 


(ii) All 2x2 matrices A such that det (A) = 0 

a b 

(iii) All the matrices of the form 

0 c 

Solution Let W is the set of all matrices 


a b 
c d 


where a + b +c + d = 0 . 


(i) Suppose u 


(where e 


e f 

g h 

(Where l + m+ n+p = 0 ) are in W. 


Then u + v 


e f 

g h 


+ 


l m 


g + h = 0 ) and v : 


e + l f + m 
g + n h + p 


n p 

Where (e + l) + (f + m) + (g + n) + ( h + p) 

= (e +f+ g + h) + (l + m + n + p) =0 + 0 = 0 
Therefore, u + v e W 

e f 

g h 
ke kf 
kg kh 

Where ke + kf+kg + kh=k(e+f+g + h)=k0 = 0 
Hence, k ue W. Therefore, W is subspace of M 22 - 


l m 
n p 


(By definition) 


If k is any scalar and u = 

1 e f 

Then k u = k\ 

g h 


(where e+f+g + h = 0) is any vector in W. 

(by definition) 


(ii) Let W is the set of all 2 x 2 matrices A such that det (A ) = 0 

e f 

g h 


Suppose u 


(Where det (u) = e h -f g = 0) and v 


/ m 
n p 
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(Where det (v) = l p - m n = 0 ) are in W. 

e +1 


Then u + v 


e f 

g h 


+ 


/ m 
n p 


g + n 


f + m 
h + p 


(By definition) 


Where det (u + v) = (e + l) (h + p) - (f + m) (g + n) 

= eh + ep + lh+lp -f g -f n - m g - m n 

= (e h -f g) + (l p - m n) + e p + l h -f n - mg = ep + lh -f n - m g ^ 0 
Therefore, u + v£ W. Therefore, W is not subspace of M 22 ■ 


(iii) Let W is the set of all matrices of the form 


a b 
0 c 


Suppose u = 

Then u + v 


e f 

0 g_ 

e f 

_0 g 


and v 


/ m 
0 n 


+ 


Which is of the fonn 


/ m 
0 n 
a b 
0 c 

e f 

0 g. 
ke kf 
0 kg 
a b 
0 c 

Therefore, W is subspace of M 2 2 - 


If k is any scalar and u = 
Then k u = k 


are in W. 

e + l f + m 
0 g + n 

. Therefore, u + v e W 
is any vector in W. 


e f 

0 g 


Which is of the fonn 


. Hence, kueW 


(By definition) 


(By definition) 


Example 14 Determine which of the following are subspaces of the space F(- 00 , 00 ) . 
(i) All / such that/ (x) <0 for all x (ii) all / such that f (0) = 0 

(iii) All/ such that f (0) =2 (iv) all constant functions 

(v) All/ of the form ki + k 2 sin x, where kj and k 2 are real numbers 

(vi) All everywhere differentiable functions that satisfy f + 2f = 0. 


Solution (i) Let W is the set of all/such that f (x) <0 for all x. 

Suppose g and h are the vectors in W. Then g (x) <0 for all x and h (x) < 0 for all x. 

Now (g + h) (x) = g (x) + h (x) < 0. Therefore, g + h e W 

If k is any scalar and g is any vector in W. Then g (x) <0 for all x 

Now (kg) (x) = kg (x), which is greater than 0 for negative real values of k. 

:.kg £ W Vk<0. 

Hence W is not the subspace ofiF(-oo,oo) . 
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(ii) Let W is the set of all/ such that f (0) = 0. 

Suppose g and h are the vectors in W. Then g (0) = 0 and h (0) = 0 
Now (g + h) (0) = g (0) + h (0) = 0 + 0 = 0. Therefore, g + h e W 
If k is any scalar and g is any vector in W. Then g (0) = 0 

Now (kg) (0) = kg (0) = k 0 = 0. :.kg e W . Hence IFis the subspace of F(-co,oo) . 

(iii) Let W is the set of all / such that f (0) =2 

Suppose g and h are the vectors in W. Then g (0) = 2 and h (0) =2 

Now (g + h) (0) = g (0) + h (0) = 2 + 2 ^ 2kg <£W . Hence W is not the subspace of 

F(-oo,oo ). 

(iv) Let W is the set of all constant functions. Suppose g and h are the vectors in W. 

Then g (x) = a and h (x) = b, where a and b are constants. 

Now (g + h) (x) = g (x) + h (x) = a + b, which is constant. Therefore, g + h eW 

If k is any scalar and g is any vector in W. Then g (x) = a, where a is any constant. 

Now (kg) (x) = kg (x) = k a, which is a constant. .'. kg e W . Hence W is the subspace 
of F(-co,co) . 

(v) Let W is the set of all / of the form Ay + Ay sin x, where Ay and Ay are real numbers 
Suppose g and h are the vectors in W. Then g (x) = m / +m 2 sin x and h (x) = n / +n 2 sin x, 
where m /, m 2 , «/and n 2 are real numbers. 

Now (g + h) (x) = g (x) + h (x) = [mi+m 2 sin x]+[ni+n 2 sinx] = (mi+ni)+(m 2 +n 2 ) sin 
x 

Which is of the form Ay + Ay sin x. Therefore, g + h eW 

If k is any scalar and g is any vector in W. Then g (x) = m j + m ? sin x, where m j and m 2 
are any real numbers. 

Now (kg) (x) = kg (x) = k [mi + m 2 sin x] = (k mi) + (k m 2 ) sin x 

Which is of the fonn Ay + Ay sin x. kg e W . Hence W is the subspace of F(-<x>, oo ) . 

(vi) Let W is the set of all everywhere differentiable functions that satisfy f' + 2f — 0. 
Suppose g and h are the vectors in W. Then g + 2g = 0 and h' + 2h = 0 

Now (g + h)' + 2(g + h) — g + h' + 2(g + h) = (g' + 2g) + (h' + 2h) =0 + 0 = 0 
Therefore, g + h e W 

If k is any scalar and g is any vector in W. Then g' + 2g = 0 
Now (kg)' + 2(kg) = kg' + 2kg = k(g' + 2g) = k.0 = 0 
kg e W . Hence W is the subspace of F(- °o, oo ) . 
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Remark Let n be a non-negative integer, and let P n be the set of real valued function of 
the form p( x ) = a 0 + cijX + a 2 x 2 +... +a n x" where a 0 ,a l ,a 2 ,...,a n are real numbers, 
thenT^ is a subspace F(- 00 , 00 ) . 


Example 15 Show that the invertible n x n matrices do not fonn a subspace of M nxn 


Solution Let W is the set of invertible matrices in M nxn . This set fails to be a subspace 
on both counts- it is closed under neither scalar multiplication nor addition. 


For example consider invertible matrices W — 


1 

2 



in M 


The matrix O.U is a 2x2 zero matrix, hence is not invertible; and the matrix U + Lhas a 
column of zeros, hence is not invertible. 


Theorem If Ax = 0 is a homogeneous linear system of m equations in n unknowns, then 
the set of solution vectors is a subspace of if". 


Example 16 Consider the linear systems 



“1 

-2 

3" 

X 


" 0 " 


"1 

-2 

3" 

X 


" 0 " 

(a) 

2 

-4 

6 

y 

= 

0 

(b) 

-3 

7 

-8 

y 

= 

0 


3 

-6 

9 

z 


0 


-2 

4 

-6 

z 


0 



“1 

-2 

3" 

X 


" 0 " 


"0 

0 

0 " 

X 


" 0 " 


-3 

7 

-8 

y 

= 

0 

(d) 

0 

0 

0 

y 

= 

0 


4 

1 

2 

z 


0 


0 

0 

0 

z 


0 


Each of the systems has three unknowns, so the solutions form subspaces of R 3 . 
Geometrically, this means that each solution space must be a line through origin, a plane 
through origin, the origin only, or all of R 3 . 

Solution fa) The solutions are x = 2s - 3t, y = s, z = t. From which it follows that 
x = 2y - 3z or x - 2y + 3z = 0 . 

This is the equation of the plane through the origin with n =(1, -2,3) as a normal vector. 

(b) The solutions are x = -5t, y = -1, z = t, which are parametric equations for the line 
through the origin parallel to the vector v =(-5, -1,1). 

(c) The solution is x = 0, y = 0, z = 0 so the solution space is the origin only, that is {0}. 

(d) The solutions are x = r, y = s, z = t. where r, s and t have arbitrary values, so the 
solution space is all R 3 . 

A Subspace Spanned by a Set: The next example illustrates one of the most common 
ways of describing a subspace. We know that the term linear combination refers to any 
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sum of scalar multiples of vectors, and Span {vi, ... , v p } denotes the set of all vectors that 
can be written as linear combinations of Vi, ..., v p . 

Example 17 Given Vi and v 2 in a vector space V, let H= Span {vi, v 2 ). Show that His 
a subspace of V. 

Solution The zero vector is in H , since 0 = Ovj + 0v 2 . To show that H is closed under 
vector addition, take two arbitrary vectors in H, say, 

u = siVi+S 2 V 2 and w = tiVi + t 2 V 2 

By Axioms 2, 3 and 8 for the vector space V. 

U + W = (SiVj +S 2 V 2 ) + (t]V, + t 2 v 2 ) 

= (si + tl) Vi + (s 2 + t 2 ) V2 


So u + w is in H. Furthennore, if c is any scalar, then by Axioms 7 and 9, 

CU = C (SiVi + s 2 v 2 ) = (csi) Vi + (cs 2 )v 2 
Which shows that cu is in H and H is closed under scalar multiplication. 

Thus H is a subspace of V. □ 

Later on we will prove that every nonzero subspace of R 3 , other than R 3 itself, is either 
Span {Vi, v 2 } for some linearly independent v 2 and v 2 or Span {v} for v ^ 0. In the first 
case the subspace is a plane through the origin and in the second case a line through the 
origin. (See Figure below) It is helpful to keep these geometric pictures in mind, even for 
an abstract vector space. 

A? 



The argument in Example 17 can easily be generalized to prove the following theorem. 
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Theorem 3 If Vi, ..., v p are in a vector space V, then Span {vi, ..., v p j is a subspace of 
V. 

We call Span {vi, ... , v p } the subspace spanned (or generated) by {vi,... , v p j. 
Given any subspace H of V, a spanning (or generating) set for H is a set {v /, v p j in H 
such that H = Span {Vi , v p j. 

Proof 


]=n 


f ]=n \ 


=0 


The zero vector is in H, since 0 = Ovi + 0v2+ ...+0v n = ^ 0V y =0 ^ V / 

j =o \j=o ) 

To show that H is closed under vector addition, take two arbitrary vectors in H, say, 


U = S!V! +S 2 V2 + ...+t„V„ = '^ j S l V l 

i=0 

and 

k-n 

w = t,v! + t 2 v 2 ...+t n v„=Y^t k V k 


k =0 


By Axioms 2, 3 and 8 for the vector space V. 


k=n 


U + W= Yj S i V i + Yj f k V k = ( S > V 1 +S2V2 + ...+S„V n ) + (tlV 1 + t 2 V 2 ...+t„V„) 


i =0 


k =0 


p=n 


= (si + tl) Vi + (s 2 + t 2 ) V 2 +...+(s n + tn) V„=X( 5 p +t p) V p 

P =0 

So u + w is in H. Furthermore, if c is any scalar, then by Axioms 7 and 9, 


CU = c(siVl +S 2 V2 + ...+S n V n )= (CSl) Vi + (CS 2 )V2 +...+(CS„)V„=^CS r V r . 

r =0 

Which shows that cu is in H and H is closed under scalar multiplication. 

Thus H is a subspace of V. □ 

Example 18 Let H be the set of all vectors of the form (a - 3b, b - a, a, b), where a and 
b are arbitrary scalars. That is, let H = {(a - 3b, b - a, a, b): a and b in R}. Show that H 
is a subspace of R 4 . 


Solution Write the vectors in H as column vectors. Then an arbitrary vector in H has 
the form 


a -3b 


' 1 ' 


"-3" 

b-a 

= a 

-1 

+ b 

1 

a 


1 


0 

b 


0 _ 


1 


T t 

Vi v 2 
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This calculation shows that H = Span {vi, V 2 }, where Vi and v? are the vectors indicated 
above. Thus H is a subspace of R 4 by Theorem 3. □ 

Example 18 illustrates a useful technique of expressing a subspace H as the set of linear 
combinations of some small collection of vectors. If H= Span {vi, ..., v p }, we can think 
of the vectors Vi, ..., v p in the spanning set as “handles” that allow us to hold on to the 
subspace H. Calculations with the infinitely many vectors in H are often reduced to 
operations with the finite number of vectors in the spanning set. 
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Exercises 


In exercises 1-13 a set of objects is given together with operations of addition and scalar 
multiplication. Detennine which sets are vector spaces under the given operations. For 
those that are not, list all axioms that fail to hold. 

1. The set of all triples of real numbers (x, y, z) with the operations 

(x, y, z) + (x, y', z) = (x + x', y + y, z + z) and k(x, y, z) = (kx, y, z) 

2. The set of all triples of real numbers (x, y, z) with the operations 

(x, y, z) + (x, y, z') = (x + x',y + y, z + z) and k(x, y, z) = (0,0,0) 

3. The set of all pairs of real numbers (x, y) with the operations 

(x, y) + (x, y) = (x + x, y + y) and k(x, y) = (2 Lx, 2ky) 

4. The set of all pairs of real numbers of the form (x, 0) with the standard operations on 
R 2 . 

5. The set of all pairs of real numbers of the form (x, y), where x> 0, with the standard 
operations on R 2 . 

6. The set of all n-tuples of real numbers of the form (x, x, x) with the standard 
operations on R". 

7. The set of all pairs of real numbers (x, y) with the operations. 

(x, y) + (x, y) = (x+x' + l,y+y' + l) and k(x, y) = (kx,ky) 


8 . The set of all 2x2 matrices of the form 
multiplication. 

9. The set of all 2x2 matrices of the form 
multiplication. 


a 1 
1 b 

a 0 
0 b 


with matrix addition and scalar 


with matrix addition and scalar 


10. The set of all pairs of real numbers of the form (1, x) with the operations 
(1, y) + (1, y') = (1, y + /) and k( 1, y) = {\,ky ) 


11. The set of polynomials of the form a + bx with the operations 

(ao + a I x) + (b 0 + bjX) = (a 0 +b 0 ) + (a, + )x and k(a„ + a,x ) = (ka 0 ) + (ka, )x 

12. The set of all positive real numbers with operations x +j = jcj and lev = 

13. The set of all real numbers (x, y) with operations 
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(x, y) + (x', /) = (xx', yy') and k( x, y) = (kx, ky) 

14. Determine which of the following are subspaces ofM„„. 

(a) all n x n matrices A such that tr (A) = 0 

(b) all n x n matrices A such that A r = -A 

(c) all n x n matrices A such that the linear system Ax = 0 has only the trivial solution 

(d) all n x n matrices A such that AB = BA for a fixed n x n matrix B 

15. Determine whether the solution space of the system Ax = 0 is a line through the 
origin, a plane through the origin, or the origin only. If it is a plane, find an equation for 


it; if it is a line, find parametric equations for it. 



'-1 

1 

r 


" 1 

-2 31 

(a) A = 

3 

-1 

0 

(b) A = 

-3 

6 

9 


2 

-4 

- 

5j 


-2 

4 -6j 


"1 

2 

3" 



"1 

2 -6] 

(c) A = 

2 

5 

3 


(d) A — 

1 

4 

4 


1 

0 

8 



3 

10 6 


16. Determine if the set “all polynomial in p n such that p(0) = 0 ” is a subspace of P„ for 
an appropriate value of n. Justify your answer. 


17. Let //be the set of all vectors of the form 


x 

3s 

2s 


. Find a vector v in « such that H = 


Span {vj. Why does this show that //is a subspace of R 3 ! 


18. Let Wbe the set of all vectors of the form 


5b + 2 c 
b 
c 


, where b and c are arbitrary. 


Find vectors u and v such that W= Span {u, v}. Why does this show that W is a subspace 


of R 3 ! 


s + 3t 


19. Let fVbe the set of all vectors of the form 


s-t 
2s -t 


. Show that W is a subspace of R 4 . 


41 
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20. Let Vj = 

~r 

0 

>*2 = 

~2" 

1 

>*3 = 

K) 4^' 
_ 1 

, and w = 

"3" 

1 


-1 


3 


6 


2 


(a) Is w in {vi, v 2 , vy/? How many vectors are in {vi, V 2 , Vy/? 

(b) How many vectors are in Span {v/, v 2 , Vy/? 

(c) Is w in the subspace spanned by {vi, v 2 , vy/? Why? 

In exercises 21 and 22, let fVbe the set of all vectors of the form shown, where a, b and c 
represent arbitrary real numbers. In each case, either find a set S of vectors that spans W 
or give an example to show that W is not a vector space. 


21 . 




a-b 

3 a + b 


b-c 

4 

22 . 

c-a 

a-5b 





_ b 


23. Show that w is in the subspace of R 4 spanned by v 1} v 2 , v 3 , where 



"-9" 


"7“ 


~-4~ 


~-9~ 


7 


-4 


5 


4 

w = 

4 

> v i = 

-2 

= 

-1 

’ V 3 = 

4 


_ 8 _ 


9 


-7 


-7 


24. Determine ify is in the subspace of R 4 spanned by the columns of A, where 


"6" 


“5 

-5 -9" 

7 

,A = 

8 

8 -6 

1 

-5 

-9 3 

-4 


_3 

-2 -7_ 
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Lecture 21 


Null Spaces, Column Spaces, and Linear Transformations 

Subspaces arise in as set of all solutions to a system of homogenous linear equations as 
the set of all linear combinations of certain specified vectors. In this lecture, we compare 
and contrast these two descriptions of subspaces, allowing us to practice using the 
concept of a subspace. In applications of linear algebra, subspaces of R n usually arise in 
one of two ways: 

> as the set of all solutions to a system of homogeneous linear equations or 

> as the set of all linear combinations of certain specified vectors. 

Our work here will provide us with a deeper understanding of the relationships between 
the solutions of a linear system of equations and properties of its coefficient matrix. 


Null Space of a Matrix 


Consider the following system of homogeneous equations: 

Xj - 3x 2 - 2x 3 = 0 


-5xj + 9 x 2 + x 3 = 0 

In matrix form, this system is written as Ax = 0 , where 


A = 


1 

-5 


-3 

9 


-2 

1 


( 1 ) 

( 2 ) 


Recall that the set of allx that satisfy (1) is called the solution set of the system (1). Often 
it is convenient to relate this set directly to the matrix .4 and the equation Ax = 0. We call 
the set of x that satisfy Ax = 0 the null space of the matrix A. The reason for this name is 
that if matrix A is viewed as a linear operator that maps points of some vector space V 
into itself, it can be viewed as mapping all the elements of this solution space of AX = 0 
into the null element "0". Thus the null space N of A is that subspace of all vectors in V 
which are imaged into the null element “0" by the matrix A. 


NULL SPACE 


Definition The null space of an m x n matrix A, written as Nul A, is the set of all 
solutions to the homogeneous equation Ax = 0. In set notation, 

Nul A = (x: x is in R" and Ax = 0} 

OR 

Nul (A) = {x / Vx g R, Ax = 0} 

A more dynamic description of Nul A is the set of all x in R" that are mapped into the 
zero vector of R m via the linear transfonnation x —» Ax , where A is a matrix of 
transfonnation. See Figure 1 
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~ 1 -3 -2] 

and let u = 

“5“ 

Let A = 

3 


-5 9 1 






-2 


. Determine if a € Nul A 


Solution To test if u satisfies Au = 0, simply compute 

Au = I II 3 1 = 1 1 = 1 . Thus u is in Nul A. 

Example Determine the null space of the following matrix: 

( 4 0 ^ 


"1 -3 -2 

“5“ 

a 


5-9 + 4 


" 0 " 

-5 9 1 

-2 


-25 + 27-2 


0 


A = 


v -8 20 j 


Solution To find the null space of A we need to solve the following system of equations: 


f 4 
-8 


0 

20 




\ X 2j 


4x 1 + 0x 2 
-8x, + 20x 


2 J 


f 0 l 

vOv 


vOy 


and 


• 4xj + 0x 2 = 0 
- 8xj + 20 x 2 = 0 


Xj = 0 

x 2 =0 


We can find Null space of a matrix with two ways i.e. with matrices or with system of 
linear equations. We have given this in both matrix form and (here first we convert the 
matrix into system of equations) equation form. In equation form it is easy to see that by 
solving these equations together the only solution is x 1 = x 2 = 0 . In terms of vectors from 

M 2 the solution consists of the single vector {0} and hence the null space of ^4 is {0} . 


Activity Detennine the null space of the following matrices: 
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'0 0 0 " 

1 . 0 = 0 0 0 
v° 0 °y 



In earlier (previous) lectures, we developed the technique of elementary row operations 
to solve a linear system. We know that perfonning elementary row operations on an 
augmented matrix does not change the solution set of the corresponding linear system 
Ax=0. Therefore, we can say that it does not change the null space of A. We state this 
result as a theorem: 

Theorem 1 Elementary row operations do not change the null space of a matrix. 

Or 

Null space N(A) of a matrix A can not be changed (always same) by changing the matrix 

with elementary row operations. 

Example Detennine the null space of the following matrix using the elementary row 
operations: (Taking the matrix from the above Example) 

(4 0 > 

A = 

{-S 20 

Solution First we transform the matrix to the reduced row echelon form: 

(4 0 W 1 0^ 1 

-R, 

{-S 20 J y-8 20 J 4 1 

fl 0 ) 

~ + 8 R, 

^0 20J 2 1 

fl 0) 1 

— R 2 

^0 1J 20 

which corresponds to the system 

x, =0 

x 2 =0 

Since every column in the coefficient part of the matrix has a leading entry that means 
our system has the trivial solution only: 

x l = 0 
x 2 = 0 

This means the null space consists only of the zero vector. 

We can observe and compare both the above examples which show the same result. 
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Theorem 2 The null space of an m x n matrix A is a subspace of R". Equivalently, the 
set of all solutions to a system Ax = 0 of m homogeneous linear equations in n unknowns 
is a subspace of R". 

Or simply, the null space is the space of all the vectors of a Matrix A of any order those 
are mapped (assign) onto zero vector in the space R" (i.e. Ax = 0). 

Proof We know that the subspace of A consists of all the solution to the system Ax = 0. 

First, we should point out that the zero vector, 0, in R" will be a solution to this system 
and so we know that the null space is not empty. This is a good thing since a vector 
space (subspace or not) must contain at least one element. 

Now we know that the null space is not empty. Consider u, v be two any vectors 
(elements) (in) from the null space and let c be any scalar. We just need to show that the 
sum (u+v) and scalar multiple (c.u) of these are also in the null space. 


Certainly Nul A is a subset of R n because A has n columns. To show that Nul(A) is the 
subspace, we have to check three conditions whether they are satisfied or not. If Nul(A) 
satisfies the all three condition, we say Nul(A) is a subspace otherwise not. 

First, zero vector “0” must be in the space and subspace. If zero vector does not in the 
space we can not say that is a vector space (generally, we use space for vector space). 

And we know that zero vector maps on zero vector so 0 is in Nul(A). Now choose any 
vectors u, v from Null space and using definition of Null space (i.e. Ax=0) 

Au = 0 and Av = 0 

Now the other two conditions are vector addition and scalar multiplication. For this we 
proceed as follow: 

Let start with vector addition: 

To show that u + v is in Nul A, we must show that A (u + v) = 0. Using the property of 
matrix multiplication, we find that 
A (u+v) = Au +Av = 0 + 0 = 0 

Thus u + v is in Nul A, and Nul A is closed under vector addition. 

For Matrix multiplication, consider any scalar , say c, 

A (cu) = c (Au) = c (0) = 0 

which shows that cu is in Nul A. Thus Nul A is a subspace of R n . 

Example 2 The set H , of all vectors in R 4 whose coordinates a, b, c, d satisfy the 
equations 

a - 2b + 5c = d 
c - a = b 

is a subspace of R 4 . 

Solution Since a - 2b + 5c = d 
c - a = b 

By rearranging the equations, we get 

a-2b + 5c-d = 0 
-a - b + c =0 
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We see that H is the set of all solutions of the above system of homogeneous linear 
equations. 

Therefore from the Theorem 2, H is a subspace of R 4 . 

It is important that the linear equations defining the set H are homogeneous. Otherwise, 
the set of solutions will definitely not be a subspace (because the zero-vector (origin) is 
not a solution of a non- homogeneous system), geometrically means that a line that not 
passes through origin can not be a subspace, because subspace must hold the zero vector 
(origin). Also, in some cases, the set of solutions could be empty. In this case, we can not 
find any solution of a system of linear equations, geometrically says that lines are parallel 
or not intersecting. 

If the null space having more than one vector, geometrically means that the lines intersect 
more than one point and must passes through origin (zero vector) . 

An Explicit Description of Nul A 

There is no obvious relation between vectors in Nul A and the entries in A. We say that 
Nul A is defined implicitly, because it is defined by a condition that must be checked. No 
explicit list or description of the elements in Nul A is given. However, when we solve the 
equation Ax = 0, we obtain an explicit description of Nul A. 


Example 3 Find a spanning set for the null space of the matrix 



"-3 

6 

-1 

1 

-7“ 


A = 

1 

-2 

2 

3 

-1 



2 

-4 

5 

8 

-4 


Solution 

The 

first 

step 

is 

to find the general solution of Ax = 0 in terms of free 


variables. 

After transforming the augmented matrix [A 0 ] to the reduced row echelon form and we 
get; 



"l 

-2 

0 -1 

3 

0 


0 

0 

1 

2 

-2 

0 


0 

0 

0 

0 

0 

0 


which corresponds to the system 

Xj - 2 x 2 - x 4 + 3x 3 = 0 

x 3 + 2 x 4 - 2 x 5 = 0 


0 = 0 

The general solution is 

x, = 2 x 2 + x 4 - 3 x 5 

x 2 = free variable 
x 3 =- 2 x 4 + 2 x 3 
x 4 = free variable 
x 5 = free variable 
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Next, decompose the vector giving the general solution into a linear combination of 
vectors where the weights are the free variables. That is, 


X; 


2x, +x 4 -3x, 


"2l 


'll 


"-3" 

x 2 


X 2 


1 


0 


0 

X, 

= 

-2 x 4 + 2x 5 

= X, 

0 

+ x 4 

-2 

+ x 5 

2 

*4 


x 4 


0 


1 


0 

x s\ 


X 5 \ 


0_ 


0_ 


1 


t t t 


U V w 


= x,u + x 4 v + x 5 w (3) 

Every linear combination of u, v and w is an element of Nul A. Thus {u, v, w} is a 
spanning set for Nul A. 


Two points should be made about the solution in Example 3 that apply to all problems of 
this type. We will use these facts later. 

1. The spanning set produced by the method in Example 3 is automatically linearly 
independent because the free variables are the weights on the spanning vectors. 
For instance, look at the 2 nd , 4 th and 5 th entries in the solution vector in (3) and 
note that x 2 u + x 4 v + x 5 w can be 0 only if the weights xj, X 4 and xs are all zero. 

2. When Nul A contains nonzero vector, the number of vectors in the spanning set 
for Nul A equals the number of free variables in the equation Ax = 0. 


Example 4 Find a spanning set for the null space of A = 


-3 2 2 

3 6 0 

-3 -2 4 


3-606 
-2 9 2 -4 


1 

-3 

4 

5 
-5 


Solution The null space of A is the solution space of the homogeneous system 
Xj - 3x 2 + 2x 3 + 2x 4 + x 5 =0 

Oxj + 3x 2 + 6 x 3 + 0x 4 - 3x 5 = 0 

2x t - 3x 2 - 2x 3 + 4x 4 + 4x 5 = 0 

3xj - 6 x 2 + Ox, + 6 x 4 + 5x 5 = 0 

-2xj + 9 x 2 + 2x 3 - 4x 4 - 5x 5 = 0 


©Virtual University Of Pakistan 


274 





21-Null Spaces, Column Spaces and Linear Transformation 


VU 


1 

-3 

2 

2 

1 

0 " 


0 

3 

6 

0 

-3 

0 


2 

-3 

-2 

4 

4 

0 


3 

-6 

0 

6 

5 

0 


-2 

9 

2 

-4 

-5 

0 


1 

-3 

2 

2 

1 

0 " 


0 

3 

6 

0 

-3 

0 

-2R, +R 3 

0 

3 

-6 

0 

2 

0 

-3R,+R 4 

0 

3 

-6 

0 

2 

0 

2R, + R 5 

0 

3 

6 

0 

-3 

0 


1 

-3 

2 

2 

1 

0 " 


0 

1 

2 

0 

-1 

0 


0 

3 

-6 

0 

2 

0 

(1/3)R 2 

0 

3 

-6 

0 

2 

0 


0 

3 

6 

0 

-3 

0 


1 

-3 

2 

2 

1 

0 " 


0 

1 

2 

0 

-1 

0 

- 3 R 2 + R 3 

0 

0 

-12 

0 

5 

0 

-3R 2 +R 4 

0 

0 

-12 

0 

5 

0 

-3R 2 +R 5 

0 

0 

0 

0 

0 

0 


1 

-3 

2 

2 

1 

0 " 


0 

1 

2 

0 

-1 

0 


0 

0 

1 

0 

- 5/12 

0 

(-1/12)R 3 

0 

0 

-12 

0 

5 

0 


0 

0 

0 

0 

0 

0 


1 

-3 

2 

2 

1 

0 " 


0 

1 

2 

0 

-1 

0 


0 

0 

1 

0 

- 5/12 

0 

12R 3 +R 4 

0 

0 

0 

0 

0 

0 


0 

0 

0 

0 

0 

0 


1 

-3 

0 

2 

11/6 

0 " 


0 

1 

0 

0 

- 1/6 

0 

- 2R , + R 1 

0 

0 

1 

0 

- 5/12 

0 

j 2 

-2R 3 + R, 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 
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10 0 2 
0 10 0 
0 0 10 
0 0 0 0 
0 0 0 0 


4/3 0 

- 1/6 0 

-5/12 0 3R 2 +R, 

0 0 

0 0 


The reduced row echelon form of the augmented matrix corresponds to the system 

1 x,+ 2 x 4 +(4/3) x 5 = 0 

lx 2 + (-1/6 )x 5 =0 

/ x s + (-5/12) x 5 = 0. 

0 = 0 
0 = 0 


No equation of this system has a fonn zero = nonzero; Therefore, the system is 

consistent. The system has infinitely many solutions: 

Xj = -2 x 4 +(-4/3) x 5 x 2 =+(1/6) x 5 x 3 =+(5/12)x 5 
x 4 = arbitrary x 5 = arbitrary 

The solution can be written in the vector form: 


c 4 = (-2,0,0,1,0) c 5 = (-4/3,1/6,5/12,0,1) 


Therefore {(-2,0,0,1,0), (-4/3,1/6,5/12,0,1)} is a spanning set for Null space of A. 


Activity: 

1. A = 


2. A = 


Find an explicit description of Nul A where: 
r 2> 5 5 3 9 ^ 

v 5 1 1 0 3 y 

r 4 1-1 0 P 

-1 -1 2-3 1 

1 1-2 0-1 
v 0 0 1 1 1 2 


The Column Space of a Matrix Another important subspace associated with a matrix 
is its column space. Unlike the null space, the column space is defined explicitly via 
linear combinations. 
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Definition (Column Space) The column space of an m x n matrix A, written as Col A, 
is the set of all linear combinations of the columns of A. If A = [aj ... a „], then 
Col .4 = Span {a j ,... , a„ } 


Since Span {d!, ..., a„ } is a subspace, by Theorem of lecture 20 i.e. if v 15 ...,v are in a 
vector space V , then Span jv,,..., v^j is a subspace of V . 

The column space of a matrix is that subspace spanned by the columns of the matrix 
(columns viewed as vectors). It is that space defined by all linear combinations of the 
column of the matrix. 


Example , in the given matrix, 

'1 1 3 ^| 

2 1 4 
A = 

3 1 5 
v 4 1 6 y 

The column space CoL4 is all the linear combination of the first (1, 2, 3, 4), the second (1, 
1, 1, 1) and the third column ( 3, 4, 5, 6). That is, CoL4 = { a-(l, 2, 3, 4) + b (l, 1, 1, 1) + 

c ( 3, 4, 5, 6) }. In general, the column space CoL4 contains all the linear 
combinations of columns of A. 


The next theorem follows from the definition of Col A and the fact that the columns of A 
are in if'". 


Theorem 3 The column space of an m x n matrix A is a subspace of R m . 


Note that a typical vector in Col A can be written as Ax for some x because the notation 
Ax stands for a linear combination of the columns of A. That is, 

Col A = {b: b = Ax for some x in R"} 

The notation Ax for vectors in Col A also shows that Col A is the range of the linear 
transfonnation x —> Ax. 


Example 6 

Find a matrix A such that W= Col A. W = < 

6a -b 

a + b 

: a,binR > 

Solution 

First, write IE as a set of linear combinations 

-la 

- 




~6~ 


-l 



"6” 


MT 

W = 

a 

1 

+ b 

1 

: a,binR 

> = Span < 

1 

? 

l 



-7 


0 



-7 


°J 


Second, use the vectors in the spanning set as the columns of A. Let A = 1 


-1 

1 

0 


Then W = Col A, as desired. 
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x 3 X? 



We know that the columns of A span if'" if and only if the equation Ax = b has a 
solution for each b. We can restate this fact as follows: 

The column space of an m x n matrix A is all of R' n if and only if the equation Ax = b has 
a solution for each b in R m . 


Theorem 4 A system of linear equations Ax = b is consistent if and only if b in the 
column space of ,4. 


Example 6 A vector b in the column space of A. Let Ax = b is the linear system 

. Show that b is in the column space of^, and express b as a 


■-1 

3 

2" 

x } 


T 

1 

2 

-3 

x 2 

= 

-9 

2 

1 

-2 

3. 


-3 


linear combination of the column vectors of A. 
Solution Augmented Matrix is given by 


"-1 

3 

2 

1 " 



1 

2 

-3 

-9 



2 

1 

-2 

-3 



"1 

-3 

-2 

-f 

1 

-1R, 

0 

5 

-1 

-8 

-ir, + r 2 

0 

7 

2 

-1 

-2R,+R 3 

"1 

-3 

-2 

-1 

- 







1/5R , 

0 

1 

-1/5 

-8/5 

2 






-7R.+R 

0 

0 

17/5 

51/5 

2 

"1 

-3 

0 

5“ 

(5/17)R } 

0 

1 

0 

-1 

(1/5)R 3 + R 2 

0 

0 

1 

3 

2R 3 + R i 
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1 0 0 
0 1 0 
0 0 1 


2 

-1 

3 


3R 2 + R, 


=>x ; = 2, x, = -l,x 3 = 3. Since the system is consistent, b is in the column space of A 



"-1~ 


~3~ 


~2" 


T 

Moreover, 2 

1 

- 

2 

+ 3 

-3 

= 

-9 


2 


1 


-2 


-3 


Example Determine whether b is in the column space of A and if so, express b as a 
linear combination of the column vectors of A: 



f 1 

1 

2^ 


r-n 

A = 

1 

0 

1 

: b = 

0 


V 2 

1 

3 ; 


V 2 J 


Solution 

The coefficient matrix Ax = b is: 


fl 

1 

2^ 

V 


'-f| 

1 

0 

1 

x 2 

= 

0 

v 2 

1 

3, 

V X 3y 


, 2 J 


The augmented matrix for the linear system that corresponds to the matrix 
equation Ax = b is: 


r l 1 2 

-r 

1 0 1 

0 

y 2 1 3 

2 J 


We reduce this matrix to the Reduced Row Echelon Fonn: 


1 2 

-r 


f l 1 2 

-n 

1 0 1 

0 

~ 

0 -1 -1 

i 

2 1 3 

2 


2 1 3 

2 

V 

/ 

V 

/ 



"l 1 2 

-r 


~ 

0 -1 -1 

i 



1° - 1 - 1 

4 . 


*,+(-!)*, 


R,+(-2)R, 


'l 1 2 

-p 

0 1 1 

-1 

\0 -1 -1 

4 J 


(-0*2 
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'l 

1 

2 

-r 


0 

1 

1 

-i 

r 3 +r 2 

1° 

0 

0 

3 y 


fl 

1 

2 

-1 

\ 

-r 3 

3 3 

0 

1 

1 

-1 

1° 

0 

0 

1 

/ 

fl 

1 

2 

-r 


0 

1 

1 

0 

R 2 +R } 

1° 

0 

0 

1 

J 


fl 

1 

2 

0 ' 


0 

1 

1 

0 

+ R^ 

1° 

0 

0 

U 


fl 

0 

1 

0 " 


0 

1 

1 

0 

*,+(-0*2 

1° 

0 

0 

U 



The new system for the equation Ax = b is 

Xj + x 3 =0 

x 2 + x 3 = 0 

0 = 1 

Equation 0=1 cannot be solved, therefore, the system has no solution (i.e. the system is 

inconsistent). 

Since the equation Ax = b has no solution, therefore b is not in the column space of A. 


Activity Determine whether b is in the column space of A and if so, express b as a 
linear combination of the column vectors of A: 

1 . 



f 1-1 2^ 



A = 

9 3 1 

; b = 

1 


U 1 b 





f 1 -1 o 


rr 

2. A = 

1 1 -1 

; b = 

2 


l-l -1 -b 


bJ 
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f 1 

1 

-2 

n 


fp 


0 

2 

0 

1 


2 

II 

rn 





; b = 



1 

1 

1 

-3 


3 


vO 

2 

2 

b 


v4. 


Theorem 5 If xq denotes any single solution of a consistent linear system Ax=b and if 
v i, v 2 ,v 3 ,...,v k form the solution space of the homogeneous system Ax=0, then every 

solution of Ax=b can be expressed in the fonn x = x 0 4- c x v 1 + c 2 v 2 +... + c k v k and, 
conversely, for all choices of scalars c l ,c 2 ,c 3 ,...,c k , the vector jc is a solution of Ax=b. 


General and Particular Solutions: The vector xo is called a particular solution of Ax=b 
.The expression xo+ Ci vi +C 2 V 2 + . . . +Ck v* is called the general solution ofy4x=b , and 
the expression C\Vi +C 2 V 2 + . . . +Ck v* is called the general solution of Ax=0. 


Example 7 Find the vector fonn of the general solution of the given linear system 
Ax = b; then use that result to find the vector form of the general solution of Ax=0. 

Xj + 3 x 2 - 2x 3 + 2x 5 = 0 

2x t + 6x 2 - 5x 3 - 2x 4 + 4x 5 - 3x 6 — -1 
5x 3 +10x 4 + 15x 6 = 5 

2x 4 + 6x 2 + 8x 4 +4x^+1 8x 6 = 6 

Solution We solve the non-homogeneous linear system. The augmented matrix of this 
system is given by 


1 

3 

-2 

0 

2 

0 

2 

6 

-5 

-2 

4 

-3 

0 

0 

5 

10 

0 

15 

2 

6 

0 

8 

4 

18 

1 

3 

-2 

0 

2 

0 

0 

0 

-1 

-2 

0 

-3 

0 

0 

5 

10 

0 

15 

0 

0 

4 

8 

0 

18 

1 

3 

-2 

0 

2 

0 

0 

0 

1 

2 

0 

3 

0 

0 

5 

10 

0 

15 

0 

0 

4 

8 

0 

18 


0 

-1 

5 

6 
0 
-1 

5 

6 
0 
1 

5 

6 


-2R l + R 1 
-2R l + R 4 


-1R 


2 
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1 3 
0 0 
0 0 
0 0 
1 3 
0 0 
0 0 
0 0 


-2 0 

1 2 

0 0 

0 0 

-2 0 

1 2 

0 0 

0 0 


2 0 

0 3 

0 0 

0 6 

2 0 

0 3 

0 6 

0 0 


0 

1 

0 

2 

0 

1 

2 

0 


-5R 2 +R 3 
-4R : + R 4 


1 3 
0 0 
0 0 
0 0 
1 3 
0 0 
0 0 
0 0 
1 3 
0 0 
0 0 
0 0 


-2 0 

1 2 

0 0 

0 0 

-2 0 

1 2 

0 0 

0 0 

0 4 

1 2 

0 0 

0 0 


2 0 

0 3 

0 1 

0 0 

2 0 

0 0 

0 1 

0 0 

2 0 

0 0 

0 1 

0 0 


0 

1 

1/3 

0 

0 

0 

1/3 

0 

0 

0 

1/3 

0 


(1/6)R 3 


-3R 3 + R 2 


2R 2 +Rj 


The reduced row echelon form of the augmented matrix corresponds to the system 

1 Xj + 3 x 2 + 4 x 4 +2 x 5 = 0 

1 x 3 +2 x 4 =0 

/ x 6 = (1/3) 

0 = 0 


No equation of this system has a fonn zero = nonzero; Therefore, the system is 

consistent. The system has infinitely many solutions: 


x 3 = -3 x, - 4 x 4 - 2 x 5 

x 2 =r 

X 3 

= -2 x. 

x 4 =s 

x 5 =t 

X 6 

= 1/3 

1 

>> 

1 

Jk 

1 

II 

x 2 = r 

X 3 

= -2 s 

x 4 =s 

*5=1 

X 6 

_ 1 


©Virtual University Of Pakistan 


282 





21-Null Spaces, Column Spaces and Linear Transformation 


VU 


This result can be written in vector form as 



which is the general solution of the given system. The vector Xo in (A) is a particular 

~-3] [-4] [-2 

1 0 0 

0-2 0 

solution of the given system; the linear combination r ^ + s ^ +/ ^ in (A) is the 

0 0 1 

_°J L 0 J L°_ 

general solution of the homogeneous system. 

Activity: 

1. Suppose that x 1 = -1, x 2 = 2, x 3 = 4, x 4 = -3 is a solution of a non-homogenous 
linear system Ax = b and that the solution set of the homogenous system Ax - 0 
is given by this formula: 
x 1 = -3 r + 4 s , 

x 2 = r-s, 
x 3 = r, 
x 4 =s 

(a) Find the vector fonn of the general solution of Ax = 0. 

(b) Find the vector fonn of the general solution of Ax = 0. 


Find the vector fonn of the general solution of the following linear system Ax = b: then 
use that result to find the vector form of the general solution of Ax=0: 


2. 


Xj - 2x, = 1 
3xj - 9 x 2 = 2 


Xj + 2x 2 - 3x 3 + x 4 = 3 
-3x! - x 2 +3 x 3 + x 4 = —1 
- Xj + 3 x 2 - x 3 + 2 x 4 = 2 
4x t -5x 2 -3 x 4 =-5 
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The Contrast between NuM and Col A 

It is natural to wonder how the null space and column space of a matrix are related. In 
fact, the two spaces are quite dissimilar. Nevertheless, a surprising connection between 
the null space and column space will emerge later. 

2 4-21 

-2-5 7 3 

3 7-86 

(a) If the column space of A is a subspace of R k , what is kl 

(b) If the null space of A is a subspace of R k , what is kl 

Solution 

(a) The columns of ,4 each have three entries, so Col A is a subspace of R k , where k = 3. 

(b) A vector x such that Ax is defined must have four entries, so Nul A is a subspace of 
R k , where k = 4. 

When a matrix is not square, as in Example 8, the vectors in Nul A and Col A live in 
entirely different “universes”. For example, we have discussed no algebraic operations 
that connect vectors in R 3 with vectors in R 4 . Thus we are not likely to find any relation 
between individual vectors in Nul A and Col A. 

2 4-2 1" 

-2 -5 7 3 , find a nonzero vector in Col A and a nonzero vector 

3 7-86 

in Nul A 

2 “ 

-2 . To 

3 

find a nonzero vector in Nul A, we have to do some work. We row reduce the augmented 

“1 0 9 0 0“ 

matrix \A 0 ] to obtain [zl 0]~ 0 1 -5 0 0 . Thus if x satisfies Ax = 0, 

0 0 0 1 0 

then x, = -9x 3 ,x 2 = 5x s , x 4 = 0 , and x$ is free. Assigning a nonzero value to xj (say), x ? = 
1, we obtain a vector in Nul A, namely, x = (-9, 5, 1, 0). 

n m 

2 4-21 

-2 

-2 -5 7 3 , let u= and v 

-1 

3 7-86 

L°_ 

(a) Determine if u is in Nul A. Could u be in Col A1 




Solution It is easy to find a vector in Col A. Any column of A will do, say, 
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(b) Determine if v is in Col A. Could v be in Nul A? 

Solution (a) An explicit description of Nul A is not needed here. Simply compute the 
product 







"3” 






~2 

4 

-2 

f 

-2 


"O' 


"0" 

Au = 

-2 

-5 

7 

3 

-1 

— 

-3 

it 

0 


3 

7 

-8 

6 

0_ 


3 


0 











Obviously, u is not a solution of Ax = 0, so u is not in Nul A. 

Also, with four entries, u could not possibly be in Col A, since Col A is a subspace of R 3 . 
(b) Reduce [A v] to an echelon form: 


[A 


"2 

4 

-2 

1 

3" 


"2 

4 

-2 

1 

3" 

-2 

-5 

7 

3 -1 

~ 

0 

1 

-5 

-4 

2 

3 

7 

-8 

6 

3 


0 

0 

0 

17 

1 


At this point, it is clear that the equation Ax = v is consistent, so v is in Col A. With only 
three entries, v could not possibly be in Nul A, since Nul A is a subspace of R 4 . 


The following table summarizes what we have learned about Nul /I and Col A. 


1. Nul A is a subspace of R n . 

2. Nul A is implicitly defined; i.e. we 
are given only a condition (Ax = 0 ) 
that vectors in Nul A must satisfy. 

3. It takes time to find vectors in Nul 
A. Row operations on [A 0\ are 
required. 

4. There is no obvious relation 
between NulT and the entries in A. 


5. A typical vector v in Nul A has the 
property that Av = 0. 

6. Given a specific vector v, it is easy 
to tell if v is in Nul A. Just compute 

Av. 

7. Nul A = {0} if and only if the 
equation Ax = 0 has only the trivial 
solution. 

8. Nul A = {0} if and only if the linear 
transformation x —> Ax is one-to- 
one. 


1. Col A is a subspace of R m . 

2. Col A is explicitly defined; that is, 
we are told how to build vectors in 
Col A 

3. It is easy to find vectors in Col A 
The columns of ,4 are displayed; 
others are formed from them. 

4. There is an obvious relation 
between Col A and the entries in 
A, since each column of A is in Col 
A. 

5. A typical vector v in Col A has the 
property that the equation Ax = v 
is consistent. 

6. Given a specific vector v, it may 
take time to tell if v is in Col A. 
Row operations on [A v] are 
required. 

7. Col A.= R'" if and only if the 
equation Ax = b has a solution for 
every b in R"'. 

8. Col A = R" 1 if and only if the linear 
transformation x —» Ax maps R" 
onto R"'. 
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Kernel and Range of A Linear Transformation 

Subspaces of vector spaces other than R" are often described in tenns of a linear 
transfonnation instead of a matrix. To make this precise, we generalize the definition 
given earlier in Segment I. 

Definition A linear transformation T from a vector space V into a vector space W is 
a rule that assigns to each vector x in V a unique vector T (x) in W, such that 

(i) T (u + v) = T (u) + T (v) for all u, v in V, and 

(ii) T ( cu ) = c T (h) for all u in V and all scalars c. 

The kernel (or null space) of such a T is the set of all u in V such that T (u) = 0 (the zero 

vector in W). The range of T is the set of all vectors in W of the form T (jc) for some x in 

V. If T happens to arise as a matrix transformation, say, T (x) = Ax for some matrix A - 
then the kernel and the range of T are just the null space and the column space of.4, as 
defined earlier. So if T(x) = Ax, c old = range of T. 


Definition If T : V —»■ W is a linear transformation, then the set of vectors in V that T 
maps into 0 is called the kernel of T; it is denoted by ker(T). The set of all vectors in W 
that are images under T of at least one vector in V is called the range of J; it is denoted 
by R(T). 

Example If T A : R" —> R"' is multiplication by the m x n matrix A, then from the 
above definition; the kernel of T 4 is the null space of A and the range of 7j is the column 
space of A. 

Remarks The kernel of T is a subspace of V and the range of J is a subspace of W. 



Kernel is a 
subspace of V 


Range is a 
subspace of W 


Figure 2 Subspaces associated with a linear transformation. 

In applications, a subspace usually arises as either the kernel or the range of an 
appropriate linear transformation. For instance, the set of all solutions of a homogeneous 
linear differential equation turns out to be the kernel of a linear transfonnation. Typically, 
such a linear transfonnation is described in terms of one or more derivatives of a 
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function. To explain this in any detail would take us too far a field at this point. So we 
present only two examples. The first explains why the operation of differentiation is a 
linear transformation. 

Example 11 Let V be the vector space of all real-valued functions / defined on an 
interval [a, h] with the property that they are differentiable and their derivatives are 
continuous functions on [a, b\. Let IF be the vector space of all continuous functions on 
[a, b] and let D:V —>W be the transfonnation that changes f in V into its 
derivative /'. In calculus, two simple differentiation rules are 
D(f+g) = D(f) + D(g)mdD(cf) = cD{f) 

That is, D is a linear transformation. It can be shown that the kernel of D is the set of 
constant functions of [a, b] and the range of D is the set W of all continuous functions on 
[a, b]. 


Example 12 The differential equation y" + wy = 0 (4) 

where w is a constant, is used to describe a variety of physical systems, such as the 
vibration of a weighted spring, the movement of a pendulum and the voltage in an 
inductance - capacitance electrical circuit. The set of solutions of (4) is precisely the 
kernel of the linear transformation that maps a function y = f(t ) into the 
function f"(t) + wf (t ). Finding an explicit description of this vector space is a problem in 
differential equations. 


Example 13 Let W 


: a - 3b - c = 0 


. Show that Wis a subspace of R in 


different ways. 

Solution First method: IF is a subspace of R 3 by Theorem 2 because IF is the set of all 
solutions to a system of homogeneous linear equations (where the system has only one 
equation). Equivalently, W is the null space of the 1x3 matrix A = [1 -3 -1]. 

Second method: Solve the equation a - 3b - c = 0 for the leading variable a in terms of 
the free variables b and c. 

f3b + cl 


Any solution has the form 


b 


, where b and c are arbitrary, and 


c 


3 b + c 


"3l 


"f 

b 

= b 

1 

+ c 

0 

c 


0 


1 


t T 


V; V 2 
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This calculation shows that W = Span/v/, vy/- Thus IT is a subspace of R 3 by Theorem 
i.e. if v v ...,v p are in a vector spaceF, then Span is a subspace ofF . We could 

also solve the equation a - 3b - c = 0 for b or c and get alternative descriptions of IF as a 
set of linear combinations of two vectors. 



“7 -3 51 


"2l 


“7“ 

Example 14 Let ,4 = 

-4 1 -5 

-5 2 -4 

, v = 

1 

-1 

, and W = 

1 

SO ^ 
_ 1 


Suppose you know that the equations Ax = v and Ax = w are both consistent. What can 
you say about the equation Ax = v + tv? 

Solution Both v and tv are in Col A. Since Col A is a vector space, v + tv must be in 
Col A That is, the equation Ac = v + w is consistent. 


Activity 

1. Let V and W be any two vector spaces. The mapping T :V —>W such that T (v) = 
0 for every v in V is a linear transfonnation called the zero transformation. Find 
the kernel and range of the zero transformation. 

2. Let V be any vector space. The mapping / : V -> V defined by I(v) = v is called 
the identity operator on V. Find the kernel and range of the identity operator. 
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Exercises 



“5" 


“5 

21 

19" 

1. Determine if w = 

-3 

is in Nul A, where A= 

13 

23 

2 


2 


8 

14 

1 


In exercises 2 and 3, find an explicit description of Nul .4, by listing vectors that span the 
null space. 






"1 

-2 

0 

4 

O' 


"1 

3 5 0" 







2. 

0 

1 4 -2 

3. 

0 

0 

1 

-9 

0 





0 

0 

0 

0 

1 


In exercises 4-7, either use an appropriate theorem to show that the given set, IF is a 
vector space, or find a specific example to the contrary. 




a 


4. < 


b 

:a + b + c = 2 



c 

> 



a 



b 

a-2b = 4c 

; 


c 

2a = c + 3d 


d 

> 



~b-2d~ 




6. 

5 + d 

:b,d real 

7. 

-a + 2b 

a-2b 


b+3d 


3a-6b 


d 





In exercises 8 and 9, find A such that the given set is Col A. 



2s + 3 1 



b-c 



r + s-2t 



2 b + c + d 


8. < 

4 r + s 

:r,s,t real 

9. < 

5c-4d 

:b,c,d real 


3 r-s-t 

> 


d 



For the matrices in exercises 10-13, (a) find k such that Nul A is a subspace of R k , and 
(b) find k such that Col A is a subspace of R k . 
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10 . 



"2 

-6" 


"7 

-2 

0" 


-1 

3 


-2 

0 

-5 

A = 

-4 

12 

11. A = 

0 

-5 

7 


_ 3 

-9 


-5 

7 

-2 


12. A= 


4 

1 


5 

1 


-2 6 0 

0 1 0 


13. ^=[1 -3 9 0 -5] 


14. Let A 


-6 

-3 


12 

6 


and w 


2 

1 


. Determine if w is in Col A. Is w in Nul A ? 



-8 

-2 

-9 


2 

15. Let A = 

6 

4 

8 

and w = 

1 


4 

0 

4 


-2 


Determine if w is in Col A. Is w in Nul A? 


16. Define T P 2 ^R 2 by T(p) = 


P(0) 

P( 1) 


T(p) 


3 

15 


For instance, if p (t) = 3 + 5t + 7t 2 , then 


a. Show that T is a linear transfonnation. 

b. Find a polynomial p in P 2 that spans the kernel of T, and describe the range of T. 


17. Define a linear transformation T\ P 2 —»i? 2 by T (p) = 


Pi 0) 
Pi 1) 


. Find polynomials pi 


and p 2 in P 2 that span the kernel of T, and describe the range of T. 


18. Let M 2x2 be the vector space of all 2x2 matrices, and define T. M 2x2 —» M 2x2 by 


T (A) = A + A 1 , where A = 


a 

c 


b 

d 


(a) Show that T is a linear transformation. 

(b) Let B be any element of M 2x2 such that B T =B. Find an A in M 2x2 such that T (A) = B. 

(c) Show that the range of T is the set of B in M 2x2 with the property that B r =B. 

(d) Describe the kernel of T. 


19. Determine whether w is in the column space of A, the null space of A, or both, where 
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-4 

0 

7 

7 


1 

-2 

-3 

1 



-8 5 

-5 2 

10 -8 

3 -2 


-2 

1 

6 

1 


0 

-2 

-3 

0 


20. Let ai, as denote the columns of the matrix .4, where 

5 12 2 0 

3 3 2 -1 -12 
8 4 4 -5 12 

2 110 -2 

(a) Explain why a 3 and as are in the column space of B 

(b) Find a set of vectors that spans Nul A 

(c) Let T: R 3 —>R 4 be defined by T (x) = Ax. Explain why T is neither one-to-one nor 
onto. 


B = \a 1 a 2 a 4 ] 
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Lecture 22 

Linearly Independent Sets; Bases 

First we revise some definitions and theorems from the Vector Space; 

Definition Let Fbe an arbitrary nonempty set of objects on which two operations are 
defined, addition and multiplication by scalars. 

If the following axioms are satisfied by all objects u, v, w in V and all scalars / and 
m, then we call V a vector space. 

Axioms of Vector Space 

For any set of vectors u, v, w in V and scalars l, m, n: 

1. u + v is in V 

2. u + v = v + u 

3. u + (v + w) = (u + v) + w 

4. There exist a zero vector 0 such that 
0 +u=u+0=u 

5. There exist a vector — u in V such that 
-u + u = 0 = u + (-u) 

6. (/ u) is in V 

7. / (u + v)= l u + l v 

8 . i n (n u) = (m n) u = n (m u) 

9. (I +m) u= I u+ m u 

10 . lu = u where 1 is the multiplicative identity 


Definition A subset W of a vector space V is called a subspace of V if W itself is a 
vector space under the addition and scalar multiplication defined on V. 

Theorem If IF is a set of one or more vectors from a vector space V, then W is 
subspace of V if and only if the following conditions hold; 

(a) If u and v are vectors in W, then it + v is in W 

(b) If k is any scalar and u is any vector in W, then k u is in W. 

Definition The null space of an in x n matrix A (Nul A) is the set of all solutions of the 
horn equation Ax = 0 

Nul A = {x: x is in R" and Ax = 0} 

Definition The column space of an m x n matrix A (Col A) is the set of all linear 
combinations of the columns of A. 

If A = [a j ... a„\, 
then 

Col A = Span { ai ,... , a n } 
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Since we know that a set of vectors S' = |v 1 ,v 2 ,v 3 ,...v p | spans a given vector space V if 

every vector in V is expressible as a linear combination of the vectors in S. In general 
there may be more than one way to express a vector in V as linear combination of vectors 
in a spanning set. We shall study conditions under which each vector in V is expressible 
as a linear combination of the spanning vectors in exactly one way. Spanning sets with 
this property play a fundamental role in the study of vector spaces. 

In this Lecture, we shall identify and study the subspace H as “efficiently” as possible. 
The key idea is that of linear independence, defined as in if". 

Definition An indexed set of vectors {vj,..., v p } in V is said to be linearly 

independent if the vector equation 

c,v 1 +c 2 v 2 +... + c p v p =0 (1) 

has only the trivial solution, i.e. a = 0, .... c p = 0. 

The set {vj, ...,v p } is said to be linearly dependent if (1) has a nontrivial solution, that is, 
if there are some weights, cj, ...,c p , not all zero, such that (1) holds. In such a case, (1) is 
called a linear dependence relation among Vi, ... , v p . Alternatively, to say that the v’s 
are linearly dependent is to say that the zero vector 0 can be expressed as a nontrivial 
linear combination of the v’s. 

If the trivial solution is the only solution to this equation then the vectors in the set are 
called linearly independent and the set is called a linearly independent set. If there is 
another solution then the vectors in the set are called linearly dependent and the set is 
called a linearly dependent set. 

Just as in if", a set containing a single vector v is linearly independent if and only if v ^ 0 . 
Also, a set of two vectors is linearly dependent if and only if one of the vectors is a 
multiple of the other. And any set containing the zero-vector is linearly dependent. 

Determining whether a set of vectors a l ,a 2 ,a 3 ,...a n is linearly independent is easy when 

one of the vectors is 0; if, say, a l = 0, then we have a simple solution to 

x x a x +x 2 a 2 + x 3 a 3 +... + x n a n = 0 given by choosing x, to be any nonzero value and putting 

all the other x’s equal to 0. Consequently, if a set of vectors contains the zero vector, it 
must always be linearly dependent. Equivalently, any set of linearly independent vectors 
cannot contain the zero vector. 

Another situation in which it is easy to determine linear independence is when there are 
more vectors in the set than entries in the vectors. If n > m, then the n vectors 
a l ,a 2 ,a 3 ,...a n in R'" are columns of an m x n matrix A. The vector equation 

x x a x + x 2 a 2 + x 3 a 2 +... + x n a n = 0 is equivalent to the matrix equation Ax = 0 whose 
corresponding linear system has more variables than equations. Thus there must be at 
least one free variable in the solution, meaning that there are nontrivial solutions 
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tox x a x +x 2 a 2 + x 3 a 2 +... + x n a n = 0: If n > m, then the set [a l ,a 2 ,a 3 ,...a n } of vectors in R" 1 
must be linearly dependent. 

When n is small we have a clear geometric picture of the relation amongst linearly 
independent vectors. For instance, the case n = 1 produces the equation x l a ] = 0, and as 
long asaj ^ 0, we only have the trivial solution x, = 0 . A single nonzero vector always 
forms a linearly independent set. 

When n = 2, the equation takes the form xp ] + x 2 a 2 = 0 . If this were a linear dependence 
relation, then one of the jc’s, say x 1 , would have to be nonzero. Then we could solve the 
equation for a 1 and obtain a relation indicating that a x is a scalar multiple ofa 2 . 
Conversely, if one of the vectors is a scalar multiple of the other, we can express this in 
the form x x a x + x 2 a 2 =0 . Thus, a set of two nonzero vectors is linearly dependent if and 
only if they are scalar multiples of each other. 

Example (linearly independent set) 

Show that the following vectors are linearly independent; 


~- 2 \ 


" 2 “ 


“ 0 " 

1 

. V 2 = 

1 

» V 3 = 

0 

1 


-2 


1 


Solution Let there exist scalars c \, c 2 , c 3 in R such that 

CjVj + c 3 v 2 + c 3 v 3 = 0 

Therefore, 



”-2] 


2] 


" 0 " 

=> C 1 

1 

+ c 2 

1 

+ c 3 

0 


1 


-2 


1 



1 

O - 

(N 

1 

i _ 


2c 2 


"0 " 


c i 

+ 

c 2 

+ 

0 


cj 


1 

1 

K) 

Ci 

to 

1 _ 


_ C 3 _ 



-2cj + 2 c 2 


" 0 " 

=> 

Cj +C 2 

= 

0 


f \ - 2 c 2 + c 3 J 


0 


The above can be written as; 


-2cj +2 c 2 = 0 .(1) => —Cj +c 2 = 0. {A) {dividing by 2 on both sides of{\ )) 

q +c 2 = 0 .(2) 

q - 2 c 2 + c 3 = 0 .(3) 
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Solving (2) and (4) implies : 


C\ + c 2 =0 
-q + c 2 = 0 


0 + 2 c, =0 


c 2 =0 


Solving (2) implies: 


c, + 0 = 0 
=> q = 0 


Solving (3) implies: 
0 + 0 + c 3 =0 
=> c 3 = 0 


=> Cj = c 2 = c 3 =0 ; scalars c ), c 2 , c 3 ei? are all zero 
:. The system has trivial solution. 

Hence the given vectors v l5 v 2 , v 3 are linearly independent. 


Example (linearly dependent set) 

Ifvj = {2,-1,0,3}, v 2 = {1,2,5,-1} and v 3 = {7,-1,5,8} , then the set of vectors 
S = {v,, v 2 , v 3 } is linearly dependent, since 3 v, +v 2 -v 3 = 0 


Example (linearly dependent set) 

The polynomials p 1 = -x + 1, p 2 = -2x 2 + 3x + 5, and p 3 = -x 2 + 3x +1 form a linearly 
dependent set in p 2 since 3p 1 - p 2 + 2 p 2 = 0. 

Note The linearly independent or linearly dependent sets can also be determined using 
the Echelon Form or the Reduced Row Echelon Fonn methods. 


Theorem 1 An indexed set { Vi, ... , v p } of two or more vectors, with v ; +0, is 
linearly dependent if and only if some Vj (with j > 1) is a linear combination of the 
preceding vectors, Vi,..., Vj.j. 

The main difference between linear dependence in R" and in a general vector space is that 
when the vectors are not n - tuples, the homogeneous equation (1) usually cannot be 
written as a system of n linear equations. That is, the vectors cannot be made into the 
columns of a matrix .4 in order to study the equation Ax = 0. We must rely instead on the 
definition of linear dependence and on Theorem 1. 

Example 1 Let pi (t) = 1, p 2 (t) = t and p 3 (t) = 4 - t. Then {p 1 , p 2 , p 3 } is linearly 
dependent in P because p 3 = 4pi —p 2 . 

Example 2 The set {Sin t, Cos t} is linearly independent in C [0, 1] because Sin t and 
Cos t are not multiples of one another as vectors in C [0, 1]. That is, there is no scalar c 
such that Cos t = c. Sin t for all t in [0, 1]. (Look at the graphs of Sin t and Cos t.) 
However, {Sin t Cos t, Sin 2t} is linearly dependent because of he identity: 

Sin 2t = 2 Sin t Cos t, for all t. 
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Useful results 

• A set containing the zero vector is linearly dependent. 

• A set of two vectors is linearly dependent if and only if one is a multiple of the 
other. 

• A set containing one nonzeoro vector is linearly independent, i.e. consider the set 
containing one nonzeoro vector jvj so jvj is linearly independent when v, ^ 0. 

• A set of two vectors is linearly independent if and only if neither of the vectors is 
a multiple of the other. 

Activity Determine whether the following sets of vectors are linearly independent or 
linearly dependent; 

1. i = (1,0,0,0), j = (0, 1,0,0 ),k = (0, 0,0,1) inM 4 . 

2 ‘ Vj = (2,0,-1), v 2 = (-3,-2,-5), v 3 = (-6,1,-1), v 4 = (-7,0,2) inK 3 . 

3. i = (1,0,0,...,0), j = (0,1,0,...,0),£ = (0,0,0,...,l) in IT . 

4. 3x 2 + 3x + 1, 4x 2 + x, 3x 2 + 6x + 5, - x 2 + 2x + 7 in p 2 

Definition Let H be a subspace of a vector space V. An indexed set of vectors B = 
/ bib,,} in V is a basis for H if 

(i) B is a linearly independent set, and 

(ii) the subspace spanned by B coincides with //; that is, 

H= Span {bi,...,b p } 

The definition of a basis applies to the case when H = V, because any vector space is a 
subspace of itself. Thus a basis of V is a linearly independent set that spans V. Observe 
that when H ^ V , condition (ii) includes the requirement that each of the vectors b i,...,b p 
must belong to H , because Span { bi,...,b p } contains bi, ...,b p , as we saw in lecture 21. 

Example 3 Let A be an invertible n x n matrix - say, A = [ai ... a„]. Then the columns 
of A form a basis for R" because they are linearly independent and they span if", by the 
Invertible Matrix Theorem. 

Example 4 Let ei,e„ be the columns of the n x n identity matrix, I n . That is, 


T 


"o' 


"o' 

0 

5 e 2 

1 

» - e „ = 

0 

o_ 


0_ 


1 


The set [e/, e n } is called the standard basis for if " (Fig. 1). 
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X5 



Xl 


Figure 1 - The standard basis for R 3 



"3” 


~-4~ 


-2 

Example 5 Let v, = 

0 

2 = 

1 

,and v 3 = 

1 


-6 


7 


5 


Determine if {vi, V 2 , V 3 } is a basis 


for R 3 . 

Solution Since there are exactly three vectors here in R 3 , we can use one of any 
methods to determine whether they are basis for M ’ or not. For this, let solve with help of 
matrices. First form a matrix of vectors i.e. matrix A = [vi v? V3J. If this matrix is 
invertible (i.e. |A| ^ 0 determinant should be non zero). 

For instance, a simple computation shows that det A = 6 V 0. Thus A is invertible. As in 
example 3, the columns of A form a basis for R 3 . 


Example 6 Let S = {1, t, 1 , ..., t n j. Verily that S is a basis for P n . This basis is called 

the standard basis for P n . 


Solution Certainly S spans P„. To show that S is linearly independent, suppose that 
co,—, c n satisfy 

Cq. 1 + Cjt + C 2 ^ + + c n t" = 0 (t) (2) 

This equality means that the polynomial on the left has the same values as the zero 
polynomial on the right. A fundamental theorem in algebra says that the only polynomial 
in P„ with more than n zeros is the zero polynomial. That is, (2) holds for all t only if 
co = ...= c n = 0. This proves that S is linearly independent and hence is a basis for P„. 
See Figure 2. 
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Figure 2 - The standard basis for P 2 

Problems involving linear independence and spanning in P n are handled best by a 
technique to be discussed later. 


Example 7 Check whether the set of vectors {(2, -3, 1), (4, 1, 1), (0, -7, 1)} is basis for 
R 3 ! 

Solution The set S = {vi, v 2 , v 3 } of vectors in R 3 spans V= R 3 if 

ClVi + C 2 V 2 + C 3 V 3 = diWl + d 2 W 2 + d3W3 (*) 

with wi = (1,0,0), tv2 = (0,1,0) , iv3 = (0,0,1) has at least one solution for every set of 
values of the coefficients dj, d 2 , d 3 . Otherwise (i.e., if no solution exists for at least some 
values of di, d 2 , S does not span V. With our vectors vi, v 2 , V3, (*) becomes 

c,(2,-3,l) + c 2 (4,1,1) + c 3 (0,-7,1) = d](l,0,0) + d 2 (0,l,0) + d 3 (0,0,l) 
Rearranging the left hand side yields 

2c I +4c,+0c 3 = ld I +0d 2 +0d 3 

-3c I +lc 2 -7c 3 =0d I +ld,+0d 3 (A) 

lc 1 +lc 2 +lc 3 = 0 d 1 + 0 d 2 +ld 3 


"2 

4 

0 " 

c i 


dj 

-3 

1 

-7 

C 2 

= 

d 2 

1 

1 

1 

- C i\ 


d 3 _ 


We now find the determinant of coefficient matrix 


2 

-3 

1 


4 

1 

1 


0 

-7 

1 


to determine whether the 


system is consistent (so that S spans V), or inconsistent (S does not span V). 
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2 4 0 

Now det -3 1-7= 2(8) - 4(4) +0 = 0 

1 1 1 

Therefore, the system (A) is inconsistent, and, consequently, the set S does not span the 
space V. 

Example 8 Check whether the set of vectors 

{-4 + 11 + 3 t 2 , 6 + 5 t + 2 t 2 , 8 + 4 t + 11 2 } is a basis for P 2 ? 

Solution The set S = {pi (t), p 2 (t), p 3 (t)} of vectors in P 2 spans V = P 2 if 

ci pi (t) + c 2 p 2 (t) + cipi (t) = diqi(t) + d 2 q 2 (t) + d 3 q 3 (t) (*) 

with q 2 (t) = 1 + Ot + 01 2 , q 2 (t) = 0 + 11 + 01 2 , q 3 (t) = 0 + 01 + 11 2 has at least 
one solution for every set of values of the coefficients di, d 2 , d 3 . Otherwise (i.e., if no 
solution exists for at least some values of di, d 2 , d 3 ), S does not span V. With our vectors 
Pi(t), P 2 (t), P3(t), (*) becomes: 

Ci (-4 + 11 + 3 t 2 ) + c 2 (6 + 51 + 21 2 ) + c 3 (8 + 4 t + 11 2 ) = 

di (1 +0t +0t 2 ) + d 2 (0 +lt +0t~) + d 3 (0 +0t + 11 2 ) 

Rearranging the left hand side yields 

(-4 ci +6 c 2 +5 c 3 )l + (1 ci +5 c 2 +4 c 3 ) t + (3 c 3 +2 c 2 +1 c 3 ) t 2 = 

(1 di +0 d 2 +0 d 3 )l + (0 di +1 d 2 +0 d 3 ) t + (0 di +0 d 2 +1 d 3 ) t 2 
In order for the equality above to hold for all values of t, the coefficients corresponding to 
the same power of t on both sides of the equation must be equal. This yields the 
following system of equations: 

-4 Cj + 6 c, + 8 c 3 =ld 1 +0d 1 +0d 3 

1 c } + 5 c 2 + 4 c 3 = 0 dj+1 d-,+0 d } (A) 

3 Cj+ 2 c 7 +1 c 3 = 0 dj+0 d 2 +1 d 3 
-4 6 8 Cj dj 

=> 1 5 4 c 2 = d 2 

3 2 1 c 3 d 3 

~-4 6 8" 

We now find the determinant of coefficient matrix 1 5 4 to determine whether the 

3 2 1 

system is consistent (so that S spans V), or inconsistent (S does not span V). 

~-4 6 8~| 

Now det 1 5 4 = -26^ 0. Therefore, the system (A) is consistent, and, 

3 2 1 

consequently, the set S spans the space V. 

The set^ = {pi (t),p 2 (t), p 3 (t)} of vectors in P 2 is linearly independent if the only 
solution of 

dpi (t) + c 2 p 2 (t) + c 3 p 3 (t) = 0 (**) 
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is ci, c 2 , c$ = 0. In this case, the sets' forms a basis for span 5*. Otherwise (i.e., if a 
solution with at least some nonzero values exists), S is linearly dependent. With our 
vectors pi (t), p 2 (t),pi (t), (2) becomes; cj (-4 + 1 1 + 3 t 2 ) + c? (6 + 51 + 2 t 2 ) + cj (8 + 
4 t + 1i) = 0 Rearranging the left hand side yields 

(-4 ci +6 c 2 +5 Cs)l + (1 ci +5 C 2 +4 c 3 ) t + (3 c 1 +2 C 2 +7 C 3 ) t 2 = 0 
This yields the following homogeneous system of equations; 


-4 c,+6 c,+8 c 3 


0 


-4 6 8 
1 5 4 

3 2 1 


As det 


-4 

1 

3 


1 Cj + 5 c 2 + 4 c 3 = 0 
3 Cj + 2 c 2 +1 c 3 = 0 
6 8 " 

5 4 = -26 5* 0. Therefore the set S = {pi (t), p 2 (t), p 3 (t)} is linearly 

2 1 


independent. Consequently, the set S forms a basis for span S. 
Example 9 The set S = 


1 0 
0 0 


1- 

[ Q 

0 

0 

-1 

0 

0 

_1 

-1 

0 

_1 


0 0 
0 1 


is a basis for the vector 


space V of all 2 x 2 matrices. 

Solution To verily that S is linearly independent, we form a linear combination of the 
vectors in S and set it equal to zero; 


Cl 


'1 

0 " 


"0 

ll 


“0 

0 ~ 


“0 

0 " 


“0 

0 " 



+ C2 



+ C3 



+ C4 



= 



0 

0 


0 

0 _ 


1 

0 


0 

1 


0 

0 


1 

_1 


1 

O 

0 

1 

O 

_1 


1 

O 

1 

O 


This gives 
linearly independent. 

To verify that S spans V we take any vector 
c 3 , and C 4 such that 

Cl 


, which implies that = C 2 = C 3 = c 4 = 0. Hence S is 


a b 
c d 


in V and we must find scalars c 1 , C 2 , 


'1 

0" 


“0 

r 


“0 

0" 


“0 

0" 


a 

b~ 


Ci c 2 


0 

0 

+ C2 

0 

0 

+ C 3 

1 

0 

+ C 4 

0 

1 

= 


d \ 


— 


c 


C 3 c 4 '_ 



a b 
c d 


We find that a = a, C 2 = b, C 3 = c, and c 4 = d so that S spans V. 

The basis S in this example is called the standard basis for M22. More generally, the 
standard basis for M mn consists of mn different matrices with a single 1 and zeros for the 
remaining entries 


Example 10 Show that the set of vectors 


0 


-1 

0 


0 

-12 


1 

-1 


is a basis for the vector space V of all 2 x 2 matrices (i.e. M 22 ). 
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Solution The set S = {vi, V 2 , vs, v 4 } of vectors in M 22 spans V = M 22 if 
C1V1 + C2V2 + C3V3 + C4V4 = diWi + d->w 2 + (I3W 3 + d 4 w 4 
(*) 


"1 

o' 


"0 

r 


"0 

O' 


"0 

0" 



, W>2 = 



, w >3 = 



, n>4 = 



0 

0 


0 

0 


1 

0 


0 

1 


has at least one solution for every set of values of the coefficients di, d 2 , d 3 , d 4 . 
Otherwise (i.e., if no solution exists for at least some values of di, d 2 , d 3 , d 4 ), S does not 
span V. With our vectors Vi, V 2 , V 3 , V 4 , (*) becomes; 



"3 

6 ' 


"0 

-ll 


"0 

- 8 ' 


'1 

0 " 

Cl 

3 

-6 

+ C2 

-1 

0 

+ C3 
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Rearranging the left hand side yields 


3c I +0c 2 +0c 3 +lc 4 6 c i - 1 c 2 -8 c 3 +0 c 4 

3 c i - 1 c 2 -12 c 3 -1 c 4 -6 c i + 0 c 2 -4 c 3 +2 c 4 

ld 1 +0d 7 +0d 3 +0d 4 0d 1 +ld 7 +0d 3 +0d 4 

0 d 2 +0 d 7 +l d 3 +0 d 4 0d 1 +0d 2 +0d 3 +ld 4 


The matrix equation above is equivalent to the following system of equations 


3c ; + 0c 7 +0c 3 +lc 4 =ld 1 +0d 7 + 0d 3 +0d 4 
6cj- lc 2 - 8c 3 +0c 4 = 0d l +ld 7 +0d 3 +0d 4 
3 c t - 1 c 7 -12 c 3 - lc 4 =0d 1 +0d 7 +ld 3 +0d 4 
-6 c 2 + 0 c, - 4 c 3 + 2 c 4 = 0d 1 +0d 7 +0d 3 +ld 4 


"3 

0 

0 

r 

Cl 


d) 

6 

-1 

-8 

0 

C 2 


d 2 

3 

-1 

-12 

-1 

C 3 


d 3 

-6 

0 

-4 

2 



d 4 


We now find the determinant of coefficient matrix A = 


3 

6 

3 

-6 


0 

-1 

-1 

0 


0 

-8 

-12 

-4 


1 

0 

-1 

2 


to determine 


whether the system is consistent (so that S spans V), or inconsistent (S does not span V). 
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Now det (A) =48*0. Therefore, the system (A) is consistent, and, consequently, the set 
S spans the space V. 

Now, the set S = {vi, v 2 , v?, V 4 } of vectors in M 22 is linearly independent if the only 
solution of cjVi + C2V2 + C3V3 + C4V4 = 0 is a, C2, C3, C4 = 0. In this case the set A 
forms a basis for span S. Otherwise (i.e., if a solution with at least some nonzero values 
exists), S is linearly dependent. With our vectors Vi, V2, V3, V4, we have 



"3 

6 ' 

+ Co 

"0 

-ll 

+ C3 

"0 

-8l 

+ C4 

'1 
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"0 

0" 

Cl 

_3 

-6 

-1 

0_ 

-12 

-4 
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Rearranging the left hand side yields 


3 Cj +0 c, +0 c 3 +1 c 4 

6 c I -1 c, - 8 c 3 +0 c 4 


~0 

0 ~ 

3 Cj-1 c 2 -12 c 3 -1 c 4 

-6 c 4 +0 c 2 - 4 c 3 + 2 c 4 


0 

0 _ 


The matrix equation above is equivalent to the following homogeneous equation. 
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As det (A) = 48 *0 

Therefore the set S = {vi, V 2 , V 3 , V 4 } is linearly independent. Consequently, the set S 
forms a basis for span S. 


Example 11 Let v, = 

'll 

-2 

- v 2 = 

1 

_1 

■ v,= 

_1 


-3 


7 


6 


and 


H = Span{Vj,v 2 ,v 3 }. 


Note that V 3 = 5vj + 3v 2 and show that Span {vj, v 2 , v 3 } = Span {v/, v 2 }. Then find a 
basis for the subspace H. 

Solution 

Every vector in Span {vi, V 2 } belongs to //because 

cj Vi + C2V2 = cj Vi + C2V2 + 0 V3 
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Now let x be any vector in H - say, x = ciVi + C 2 V 2 + C 3 V 3 . Since V 3 = 5vi + 3 v 2 , we may 
substitute 

X = C 1 V 1 + C 2 V 2 + C3 (5vi + 3 V 2 ) 

= (a + 5c 3 ) Vi + (c 2 + 3c 3 ) v 2 

Thus x is in Span {vi, V 2 }, so every vector in H already belongs to Span {vi, V 2 }. We 
conclude that H and Span {vi, V2} are actually the same set of vectors. It follows that 
{v 1 , v 2 } is a basis of H since {v/, v 2 } is obviously linearly independent. 

Activity Show that the following set of vectors is basis for M ' : 


1 . 

L =(l, 0, 0), v 2 =(0, 2, l), Vj =(3, 0, l) 

2 . 

vj =(1,2, 3), v 2 =(0,1,1), Vl =(0,1, 3) 

The Spanning Set Theorem 

As we will see, a basis is an “efficient” spanning set that contains no unnecessary vectors. 
In fact, a basis can be constructed from a spanning set by discarding unneeded vectors. 

Theorem 2 (The Spanning Set Theorem) Let S = {vi, ... , v p j be a set in V and let 
H = Span {v 1 , v p }. 

a. If one of the vectors in S - say, v* - is a linear combination of the 
remaining vectors in S, then the set fonned from S by removing v* still 
spans H. 

b. If H ^ {0}, some subset of A is a basis for H. 
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Since we know that span is the set of all linear combinations of some set of vectors and 
basis is a set of linearly independent vectors whose span is the entire vector space. The 
spanning set is a set of vectors whose span is the entire vector space. "The Spanning set 
theorem" is that a spanning set of vectors always contains a subset that is a basis. 

Remark Let V = R"' and let S = {vi, V 2 ,..., v„} be a set of nonzero vectors in V. 

Procedure 

The procedure for finding a subset of S that is a basis for W = span S is as follows; 

Step 1 Write the Equation, 

dVi + C 2 V 2 + ... + c„ v n =0 (3) 

Step 2 Construct the augmented matrix associated with the homogeneous system of 
Equation (1) and transforms it to reduced row echelon form. 

Step 3 The vectors corresponding to the columns containing the leading 1 ’s form a basis 
for W = span S. 

Thus if 5* = {vi, V 2 , .... V6} and the leading l’s occur in columns 1, 3, and 4, then { Vi, v 3 , v 4 } is 
a basis for span S. 

Note In step 2 of the procedure above, it is sufficient to transform the augmented matrix to row 
echelon form. 

Example 12 Let S = {vi, V 2 , v 3 , v 4 , V 5 } be a set of vectors in R 4 , where 
v 7 = (1,2,-2,1), v 2 = (-3,0,-4,3), v 3 = (2,1,1,-1), v 4 = (-3,3,-9,6), and v 5 = (9,3,7-6). 

Find a subset of S that is a basis for W = span S. 

Solution Step 1 Fonn Equation (3), 

C] (1,2, -2,1) + c 2 (-3,0,-4,3) + c3(2,1 ,1,-1)+ c 4 (-3,3,-9,6) + c 5 (9,3.7,-6) = (0,0,0,0). 

Step 2 Equating corresponding components, we obtain the homogeneous system 
c, - 3c , + 2c 3 - 3c 4 + 9c 5 = 0 

2c j + c 3 + 3 c 4 + 3c 5 — 0 
-2c, - 4c 2 + c 3 - 9 c 4 +7c 5 = 0 
Cj + 3c, - c 3 + 6 c 4 -6c 5 =0 

The reduced row echelon form of the associated augmented matrix is 
1 0 y 2 3/2 3/2 : 0 

0 1 -1/2 3/2 -5/2 ; 0 

0 0 0 0 0:0 

0 0 0 0 0:0 

Step 3 The leading l’s appear in columns 1 and 2, so {vi, V 2 } is a basis for W = span S. 


©Virtual University Of Pakistan 


304 





22- Linear Independence Sets; Bases 


VU 


Two Views of a Basis When the Spanning Set Theorem is used, the deletion of 
vectors from a spanning set must stop when the set becomes linearly independent. If 

an additional vector is deleted, it will not be a linear combination of the remaining 
vectors and hence the smaller set will no longer span V. Thus a basis is a spanning set 
that is as small as possible. 

A basis is also a linearly independent set that is as large as possible. If S is a basis for V, 
and if S is enlarged by one vector - say, tv - from V, then the new set cannot be linearly 
independent, because S spans V, and w is therefore a linear combination of the elements 
in S. 

Example 13 The following three sets in R 3 show how a linearly independent set can be 
enlarged to a basis and how further enlargement destroys the linear independence of the 
set. Also, a spanning set can be shrunk to a basis, but further shrinking destroys the 
spanning property. 
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Linearly independent A basis 
but does not span R 3 for if 3 


Spans R 3 but is 
linearly dependent 
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V 



Example 14 Let v, = 

0 

>*2 = 

1 

, and H = < 

s 

; sin if 

>. then every vector in H is a 


0 


0 


0 





s 


"ll 


"O' 


linear combination of v 2 and v 2 because 

s 

= s 

0 

+ 5 

1 

• Is {Vy, v 2 } a basis for HI 


0 


0 


0 



Solution Neither vy nor v 2 is in H, so {vi, v 2 } cannot a basis for H. In fact, {v 2 , v 2 } is a 
basis for the plane of all vectors of the form (c j, c 2 , 0), but H is only a line. 


Activity Find a Basis for the subspace W in BL spanned by the following sets of 
vectors; 


1. Vj =(1,0,2), v 2 =(3,2,1), v 3 =(1,0,6), v 4 =(3,2,1) 

2. Vj =(1,2,2), v 2 =(3,2,1), v 3 =(1,1,7), v 4 =(7,6,4) 
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Exercises 


Determine which set in exercises 1-4 are bases for R 2 or R 3 . Of the sets that are not bases, 
determine which one are linearly independent and which ones span R 2 or R 3 . Justify your 
answers. 
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-5 

4. 

-4 
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3 

y 

-5 

y 
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-3 


6 


3 


-1 


4 


-2 


5. Find a basis for the set of vectors in R 3 in the plane x + 2y + z = 0. 

6. Find a basis for the set of vectors in R 2 on the line y = 5x. 

7. Suppose R 4 = Span {vy, V 2 , vy, Vy}. Explain why {vy, V 2 , vy, vy} is a basis for if 4 . 


8. Explain why the following sets of vectors are not bases for the indicated vector spaces. 
(Solve this problem by inspection). 

(a) ui = (1, 2), u 2 = (0, 3), us = (2, 7) fori? 2 

(b) iiy = (-l,3,2),in = (6, 1, 1) for R 3 

(c) pi = 1 + x + x 2 , p 2 = x - 1 for P 2 


(d) A = 


1 1 
2 3 


B = 


6 

-1 



3 0 
1 7 


D = 


5 1 
4 2 



1 

9 


for M 2 2 


9. Which of the following sets of vectors are bases for R 2 ! 

(a) (2, 1), (3, 0) (b) (4, 1), (-7, -8) (c) (0, 0), (1, 3) (d) (3, 9), (-4, -12) 

10. Let Kbe the space spanned by Vy = Cos x, v 2 = Sin x, vs = cos 2x. 

(a) Show that S = {vy, v 2 , Vy} is not a basis for V (b) Find a basis for V 


In exercises 11-13, determine a basis for the solution space of the system. 


Xj +x 2 - x 3 =0 


2xj + x 2 + 3 x 3 = 0 


11 . -2x, - x 2 + 2x 3 = 0 


12. Xj + 5 x 3 = 0 


- x, + x 3 =0 


x 2 + x 3 =0 
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x + y+z — 0 

3x + 2y - 2z — 0 

13. 

4x + 3y- z = 0 
6x + 5y+ z = 0 

14. Determine bases for the following subspace of R 3 

(a) the plane 3x - 2y + 5z = 0 (b) the plane x - y = 0 

(c) the line x = 2t, y = -t, z = 4t (d) all vectors of the form (a, b, c), where b = a + c 

15. Find a standard basis vector that can be added to the set {vy, V 2 } to produce a basis 
for R 3 . 

(a) Vy = (-1, 2, 3), v 2 = ( 1 , - 2 , -2) (b) Vy = (1, -1, 0), v 2 = (3, 1, -2) 


16. Find a standard basis vector that can be added to the set {vy, V 2 } to produce a basis 
for R 4 . 

Vy = (1, -4, 2, -3), V 2 = (-3, 8, -4, 6) 
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Lecture No.23 

Coordinate System 

OBJECTIVES 


The objectives of the lecture are to learn about: 

• Unique representation theorem. 

• Coordinate of the element of a vector space relative to the basis B. 

• Some examples in which B- coordinate vector is uniquely determined using basis 
of a vector space. 

• Graphical interpretation of coordinates. 

• Coordinate Mapping 


Theorem 

Let B = \b x ,b 2 ,...,b n }be a basis for a vector space V. Then for each x in V, 
there exist a unique set of scalars c 1 ,c 2 ,...,c n such that 

x=c 1 b l ,c 2 b 2 ,...,c n b n .(1) 


Proof 

Since B is a basis for a vector space V, then by definition of basis every element 
of V can be written as a linear combination of basis vectors. That is ifx e V , then 
x =c 1 b l ,c 2 b 2 ,...,c n b n . Now, we show that this representation for x is unique. 

For this, suppose that we have two representations for jr. 
i.e. 


x=c l b l ,c 2 b 2 ,...,c n b n .(2) 

and 


x=d l b l ,d 2 b 2 ,...,d n b n .(3) 

We will show that the coefficients are actually equal. To do this, subtracting (3) from (2), 
we have 


0 = (c, -flj )b t +(c 2 -d 2 )b 2 +... + (c„ - d n )b n . 

Since B is a basis, it is linearly independent set. Thus the coefficients in the last linear 
combination must all be zero. That is 


c, = d ,,..., c = d . 

1 I?? n n 

Thus the representation for x is unique. 


Definition (B-Coordinate of* ) 

Suppose that the set B = {b x ,b 2 ,...,b n } is a basis for V and x is in V. The coordinates of x 
relative to basis B (or the 5-coordinate ofx ) are the weights c l ,c 2 ,...,c n such that 

x=c l b 1 ,c 2 b 2 ,...,c n b n . 
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Note 


If c ] ,c 2 ,...,c n are the B- coordinates ofx, then the vector inf?", [jc] 


coordinate vector of x (relative to B) or B- coordinates ofx . 



Example 1 


, , , , r 11 rr 

Consider a basis B = \b x ,b 2 \ for R~ , where b x = andh 2 = ^ . 

r_2i 

Suppose an x in R 2 has the coordinate vector [jc] b = ^ . Find jc 

Solution 

Using above definition jc is uniquely detennined using coordinate vector and 
the basis. That is 


x = c t b t ,c 2 b 2 
= (-2)b t +(3)b 2 



Let S = {vi, V 2 , vj} be the basis fori?' 5 , where v/ = (l, 2, 1), V 2 = (2, 9, 0), 
and Vi = (3, 3, 4). 

(a) Find the coordinates vector of v = (5, -1, 9) with respect to S. 

(b) Find the vector v in R 3 whose coordinate vector with respect to the basis S is 

Ms-(-1,3, 2) 

Solution 


Further 


Since S is a basis for R 3 , Thus 

x = c : v, +c 2 v 2 +c 3 v 3 . 
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(5, -1,9)= Cj (1,2,1) + c 2 (2,9,0) + c 3 (3,3,4).(A) 

To find the coordinate vector of v, we have to find scalars c x , c 2 , c 3 . 
For this equating corresponding components in (A) gives 


Cj + 2c, +3 c 3 — 5 

(1) 

2c j + 9c, + 3 c 3 = -1 

(2) 

+ 

O 

Uj 

II 

(3) 


Now find values of c,, c, and c , from these equations. 
From equation (3) 

c i= 9-4c 3 

Put this value of c, in equations (1) and (2) 

9 — 4c } + 2c 2 + 3 c 3 = 5 
2ct-c 3 =-4 (4) 

and 

2(9 — 4c 3 ) + 9c 2 + 3 c 3 = -1 

18 - 8c 3 + 9c 2 + 3c 3 = -1 

9c 2 - 5c 3 =-19 (5) 

Multiply equation (4) by 5 
10 c 2 -5c 3 = -20 

Subtract equation (5) from above equation 
10c 2 -5c 3 = -20 
+Pc, + 5c 3 = +19 


c 2 =~ 1 

Put value of c 2 in equation (4) to get c 3 
2(-1) — c 3 = -4 
-2 ~c 3 = -4 
c 3 =4-2 = 2 

Put value of c 3 in equation (3) to get c, 
c 3 +4 (2)= 9 
c 3 =9-8 = 1 

Thus, we obtain ci = 1, c 2 = -1, cs =2 
Therefore, [v] s = (1,-1, 2) 

Using the definition of coordinate vector, we have 
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V =c 1 v 1 +c 2 v 2 +c 3 v 3 
= (-l)Vj + 3 v 2 + 2v 3 
= (-!)(!, 2,1) + 3(2,9,0) + 2(3,3,4) 

= (-1 + 6 + 6,-2 + 27 + 6,-1 + 0 + 8) 
= (11,31,7) 

Therefore 

v = (11,31,7) 


Example 3 

Find the coordinates vector of the polynomial p = (to + ajx + a 2 x 2 relative to 
the basis S = {/, x, x 2 } for p 2 . 

Solution 


To find the coordinator vector of the polynomial p , we write it as a linear combination of 
the basis set S . That is 

a 0 + a x x + a 2 x 2 = c l (1) + c 2 (x) + c 3 (x 2 ) 

=^> c x = a 0 , c, = a l , c 3 = a 2 

Therefore 

bl =(« 0 > a P a 2 ) 


Example 4 


Find the coordinates vector of the polynomial p = 5- 4x + Jx 2 relative to the 
basis S = {l,x, x 2 } for p 2 . 

Solution 


To find the coordinator vector of the polynomial p , we write it as a linear combination of 
the basis set S . That is 

5-4x + 3x 2 = Cj(l) + c 2 (x) + c 3 (x 2 ) 


Therefore 


5,c 2 4, c 3 3 


[p] s =(5,-4,3) 


Example 5 

Find the coordinate vector of A relative to the basis S = {Al, A2, A3, A4} 



"2 O' 


M r 


ri ii 
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o 
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"0 0" 

A = 

3J 

;4 = 

i 

o 

o 

1_ 

, A 2 

i 

o 

o 

i_ 

, 

1 0_ 


0 1 


Solution 

To find the coordinator vector of^l, we write it as a linear combination of the 
basis set S . That is 


©Virtual University Of Pakistan 


311 





23- Coordinate System 


VU 


A — ci Aj+ C2 A2+ cj Aj+ C4 A4 
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_1 
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3 
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0 < 

"4 _ 



—Cj + c, + 0 + 0 c^+c^+O + O 

O + O + Cj+0 0 + 0 + 0 + 


"2 

O' 


-c,+c 2 c,+c 2 

-1 

3 


c 3 c 4 


-c, +c, = 2 

(1) 

II 

+ 

( 2 ) 

C 3 = ~ 1 

(3) 

c 4 = 3 

(4) 


Adding (1) and (2), gives 


2 c 2 = 2 => C 2 ~ 1 

Putting the value of c? in (2) to get c /, cj = -l 

So Cl = -1, C 2 = 1, C 3 = -1, C 4 = 3 

Therefore, [v] s = (-1,1,-1, 3) 

Graphical Interpretation of Coordinates 

A coordinate system on a set consists of a one-to-one mapping of the points in the 
set into R". For example, ordinary graph paper provides a coordinate system for the plane 
when one selects perpendicular axes and a unit of measurement on each axis. Figure 1 
shows the standard basis \e h e 2 ), the vectors 
bi (= ei ) and 62 from Example 1, that is, 



'll 


" 1 " 

b, = 


and b 1 = 


1 

0_ 

2 

2 


Vector x = 



, the coordinates 1 and 6 give the location of x relative to the standard 


basis: 1 unit in the ei direction and 6 units in the t ’2 direction. 
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Figure 2 shows the vectors bi, b 2 , and x from Figure 1. (Geometrically, the three 
vectors lie on a vertical line in both figures.) However, the standard coordinate grid was 
erased and replaced by a grid especially adapted to the basis B in Example 1. The 


coordinate vector [jc]^ 


-2 

3 


gives the location of x on this new coordinate system: 


2 


units in the bi direction and 3 units in the />? direction. 



Figure 2 


Example 6 

In crystallography, the description of a crystal lattice is aided by choosing a basis 
{u, v, w) for R 3 that corresponds to three adjacent edges of one “unit cell” of the crystal. 
An entire lattice is constructed by stacking together many copies of one cell. There are 
fourteen basic types of unit cells; three are displayed in Figure 3. 
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Thus, Ci = 3, C 2 = 2. 

(Equation (3) can also be solved by row operations on an augmented matrix. Try it 
yourself ) 

Thus* = 3bi + 2 b2 and \_x~\ B = 




Figure 4 - The ^-coordinate vector of x is (3,2) 

The matrix in (3) changes the ^-coordinates of a vector x into the standard coordinates 
for x. An analogous change of coordinates can be carried out in if" for a basis 
B = {bi ,b 2 , ... , b n ). 

Let P K = [b, b 2 ... b„\ 

Then the vector equation x = c I b 1 +c,b 2 + ...+c n b n 

is equivalent to x = P /S [ x ]n (4) 

We call Pb the change-of-coordinates matrix from B to the standard basis in if". 
Left-multiplication by Pg transforms the coordinate vector [a'] /; into x. The change-of- 
coordinates equation (4) is important and will be needed at several points in next lectures. 
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Since the columns of Pb form a basis for if", Pb is invertible (by the Invertible Matrix 
Theorem). Left-multiplication by P B 1 converts x into its ^-coordinate vector: 

P B x — [.r]^ 

The correspondence x —»[x] g produced here by P B , is the coordinate mapping 

mentioned earlier. Since P B 1 is an invertible matrix, the coordinate mapping is a one-to- 

one linear transformation from R" onto if", by the Invertible Matrix Theorem. (See also 
Theorem 3 in lecture 10) This property of the coordinate mapping is also true in a general 
vector space that has a basis, as we shall see. 

The Coordinate Mapping Choosing a basis B = {bi, b 2 , ... , b n } for a vector space V 
introduces a coordinate system in V. The coordinate mapping x —»[.v] /; connects the 

possibly unfamiliar space V to the familiar space if". See Figure 5. Points in V can now 
be identified by their new “names”. 



Figure 5 - The coordinate mapping from V onto if" 


Theorem 2 Let B = {bi, b 2 , ... , b„} be a basis for a vector space V. Then the 
coordinate mapping x —> [_v] /; is a one-to-one linear transfonnation from V onto if". 
Proof Take two typical vectors in V, say 
u = c l b,+c 2 b 2 +... + c n b n 


w = d,b l +d 2 b 2 +... + d n b n 

Then, using vector operations, u + w = {c 1 +d I )b l + (c 2 + d 2 )b 2 + ... + (c„ +d„)b„ 



1 

+ 

_1 




d ] 

It follows that [u + w~\ B = 

_ c « +d «\ 

— 

_ C «J 

+ 

d„_ 


Thus the coordinate mapping preserves addition. If r is any scalar, then 

ru = r(c ] b l + c 2 b 2 + • • • + c n b n ) = (rc x )b, + (rc 2 )b 2 +••• + (rc n )b n 
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rc x 


c i 

So 

[ru] B = 

_ rC n\ 

= r 

_ C n_ 


= r[u] B 


Thus the coordinate mapping also preserves scalar multiplication and hence is a linear 
transformation. It can be verified that the coordinate mapping is one-to-one and maps V 
onto R". 


The linearity of the coordinate mapping extends to linear combinations, just as in lecture 
9. If Ui, u 2 , — , Up are in V and if cj , c 2 , ... , c p are scalars, then 
[CjUj +c 2 u 2 +... + c p u p ] B =c I [u 1 ] B +c 2 [u 2 ] B +... + c p [u p ] B (5) 

In words, (5) says that the ^-coordinate vector of a linear combination of Uj, u 2 , ... , u p 
is the same linear combination of their coordinate vectors. 


The coordinate mapping in Theorem 2 is an important example of an isomorphism from 
V onto R". In general, a one-to-one linear transformation from a vector space V onto a 
vector space W is called an isomorphism from V onto W (iso from the Greek for “the 
same”, and morph from the Greek for “form” or “structure”). The notation and 
terminology for V and W may differ, but the two spaces are indistinguishable as vector 
spaces. Every vector space calculation in V is accurately reproduced in W, and vice versa. 


Example 8 Let B be the standard basis of the space P 3 of polynomials; that is, let B = 
{1, t, t 2 , t 3 }. A typical element p of P 2 has the form p (t) = a 0 + aj t + a 2 t 2 + a 2 t 3 
Since p is already displayed as a linear combination of the standard basis vectors, we 


conclude that [P] B = 


. Thus the coordinate mapping p—>[p\ B is an isomorphism 


from Pj onto R 4 . All vector space operations in P 2 correspond to operations in R 4 . 


If we think of P 2 and R 4 as displays on two computer screens that are connected via the 
coordinate mapping, then every vector space operation in P 2 on one screen is exactly 
duplicated by a corresponding vector operation in R 4 on the other screen. The vectors on 
the P 2 screen look different from those on the R 4 screen, but they “act” as vectors in 
exactly the same way. See Figure 6. 
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Figure 6 - The space P 3 is isomorphic to R 4 


2 2 

Example 9 Use coordinate vector to verify that the polynomials 1+21, 4 + t + 5t 
and 3 + 2t are linearly dependent in P 2 . 

Solution The coordinate mapping from Example 8 produces the coordinate vectors (1, 
0, 2), (4, 1, 5) and (3, 2, 0), respectively. Writing these vectors as the columns of a 
matrix A, we can determine their independence by row reducing the augmented matrix 



"l 

4 

3 

0“ 


"1 

4 

3 

0" 

for Ax = 0: 

0 

1 

2 

0 

~ 

0 

1 

2 

0 


2 

5 

0 

0 


0 

0 

0 

0 


The columns of A are linearly dependent, so the corresponding polynomials are linearly 
dependent. In fact, it is easy to check that column 3 of A is 2 times column 2 minus 5 
times column 1. The corresponding relation for the polynomials is 
3+2t = 2(4 + t + 5t 2 ) - 5(1 + 21 2 ) 



3 


-1 


3 

Example 10 Let v, = 

6 

- V 2 = 

0 

, x = 

12 


2 


1 


7 


and B = {vj, v 2 }. Then B is a 


basis for H = Span {vi, v 2 }. Determine if x is in H and if it is, find the coordinate vector 
of x relative to B. 


Solution If x is in H, then the following vector equation is consistent. 


'3' 


'-f 


" 3" 

6 

+ c. 

0 

= 

12 

2 


1 


7 


The scalars, c 1 and c 2 , if they exist, are the B - coordinates of x. 
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"3-1 3 ” 


"1 0 2" 

Using row operations, we obtain 

6 0 12 

~ 

0 1 3 


2 1 7 


0 0 0 


Thus a 


= 2, C 2 = 3 and [:t] g = 


2 

3 


The coordinate system on H determined by B is 


shown in Figure 7. 



Figure 7 - A coordinate system on a plane //in R 3 

If a different basis for H were chosen, would the associated coordinate system also make 
//isomorphic to R 2 ! Surely, this must be true. We shall prove it in the next lecture. 



"ll 


~-3" 


" 31 


~-8" 

Example 11 Let b, = 

1 

o o 

1_ 

, b 2 

4 

0 

» b 3 = 

-6 

3 

, and x = 

2 

3 


a. Show that the set B = {bi, b 2 , bi) is a basis of/?'*. 

b. Find the change-of-coordinates matrix from B to the standard basis. 

c. Write the equation that relates x in R 3 to [_v]«. 

d. Find [.v] B , for the x given above. 


Solution 

a. It is evident that the matrix P B = [bj b 2 b$\ is row equivalent to the 

identity matrix. By the Invertible Matrix Theorem, P B is invertible and its 
columns form a basis for R 3 . 
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b. 


c. 


From part (a), the change-of-coordinates matrix is 


-3 

4 

0 


3 

-6 

3 




d. To solve part (c), it is probably easier to row reduce an augmented matrix 
instead of computing P ~ 1 . We have 


"1 

-3 

3 

-8" 


'1 

0 

1 

in 

o 

0 

4 

-6 

2 

~ 

0 

1 

0 

2 

0 

0 

3 

3 


0 

0 

l 

1 


P B x I [-*■]# 


Hence [x] B 


-5 

2 

1 


Example 12 The set B = {1 + t , 1 + f 2 , t + f 2 } is a basis for P 2 . Find the coordinate 
vector of pit) = 6 + 3t -1 2 relative to B. 

Solution The coordinates of p (t) = 6 + 3t-1 2 with respect to B satisfy 
c 1 (l + t) + c 2 (l + t 2 ) + c 3 (t + t') = 6 + 3t-t 2 

Cj + Cjt + c 2 + c 2 t + c 2 t + c 2 t = 6 + 3 1 — t 
Cj + c 2 + c y t + c 3 t + c 2 t 2 + c 3 t 2 =6 + 3 t — t 2 

C\ c 2 + (Cj + c 3 )t + (c 9 + Cj )t = 6 + 3 1 — t 

Equating coefficients of like powers of t, we have 


c,+c 2 = 6 -(1) 

Cj +c 3 = 3 -(2) 

C 2 + C 3 = - 1 -( 3 ) 


Subtract equation (2) from (1 )we get 
c 2 —c 3 = 6 - 3 = 3 

Add this equation with equation (3) 
2c 2 =-1 + 3 = 2 
c 2 ~ 1 

Put value of c 2 in equation (3) 
l + c 3 = -1 
^> c 3 =-2 
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From equation (1 ) we have 


c,+ c 2 = 6 
Cj = 6 -1 = 5 


Solving, we find that cj = 5, C 2 = 1, c 2 = -2, and [p] B = 


5 

1 

-2 


Exercises 

In exercises 1 and 2, find the vector x determined by the given coordinate vector [x]« and 
the given basis B. 


1 B = 


3 

-5 




T 


“5“ 


4" 


“3" 

2 B = 

-4 

f 

2 

> 

-7 

u[*] B = 

0 


_3 _ 


-2 


0 


-1 


In exercises 3-6, find the coordinate vector [x]« of x relative to the given basis B = {bi, 
b2t • • •; b n } . 


3. b, = 

"1" 

_-3_ 

,b 2 = 

1 1 

1 1 

,x = 

'-2 

1 



4. b : = 

-2 

>&2 

1 1 

i_i 

,X = 

1 1 

O 4^ 
l_1 


"1" 


"-3" 


"2" 


"8“ 







5. = 

-1 

,b 2 = 

4 

ii 

*C5 

-2 

,x = 

-9 








_-3_ 


9 


4 


6 









"f 


"2" 


"1" 


"3" 

6. b 2 = 

0 

> b 2 = 

1 

ii 

»© 

-1 

,x = 

-5 


_3_ 


_8_ 


2 


4 


In exercises 7 and 8, find the change of coordinates matrix from B to standard basis in 
R". 


7. 



8. B = \ 


3 

-1 

4 


2 

0 

-5 


8 

-2 
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In exercises 9 and 10, use an inverse matrix to find [.v]g for the given x and P 

10. B = 


r 

“4" 

“6“ 

1 

“2“ 

9. B = \ 



>,x = 


1 

_5_ 

[ 7_ 

1 

0 


3 

-5 


-4 

6 


-,x = 


2 

-6 


11. The set B = {1 + t 2 , t + t 2 , 1 + 2t + t 2 } is a basis for P?. Find the coordinate vector of 
p (t) = / + At + It 2 relative to B. 



~r 


" 2 " 


"-3" 

12. The vectors Vj = 

-3 

^2 = 

-8 

^3 = 

7 


two different ways to express 


span R but do not form a basis. Find 
as a linear combination of Vi,V 2 ,Vs. 


13. Let B = 


1 

-4 


-2 

9 


. Since the coordinate mapping determined by B is a linear 


transfonnation from R 2 into R 2 , this mapping must be implemented by some 2x2 matrix 
A. Find it. 

In exercises 14-16, use coordinate vectors to test the linear independence of the sets of 
polynomials. 


14. 1 + t 3 , 3 + t - It 2 , -1 + 3 1 2 -1 3 


15. (t-1) 2 , t 3 - 2, (t - 2) 3 


16. 3 + It, 5 + t- 2t*, t - It 2 , 1 + 16* - 6 t 2 + 21 3 

17. Let H= Span {vi, V 2 } and B = {vi, V 2 }. Show that* is in Hand find theP- 

coordinate vector of x, for v, 

18. Let H= Span {vi, V 2 , v^} and B = {vy, V 2 , Vy}. Show that B is a basis for H and x is 

in H, and find the P-coordinate vector of x, for v, 


"If 


14” 


"19" 

-5 


-8 


-13 

10 

^2 = 

13 

,x = 

18 

_ 7 _ 


10 


_ !5 _ 


"-6" 


1 

00 
_1 


“-9" 


1 

l_ 

4 


-3 


5 


7 

-9 

^2 = 

7 

11 

1 

OO 

11 

X 

1 

00 

1 

_1 


1 

1 

1_ 


1 

U) 

1_ 


1 

1_ 
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Lecture 24 

Dimension of a Vector Space 

In this lecture, we will focus over the dimension of the vector spaces. The 
dimension of a vector space V is the cardinality or the number of vectors in the basis B of 
the given vector space. If the basis B has n (say) elements then this number n (called the 
dimension) is an intrinsic property of the space V That is it does not depend on the 
particular choice of basis rather, all the bases of V will have the same cardinality. Thus, 
we can say that the dimension of a vector space is always unique. The discussion of 
dimension will give additional insight into properties of bases. 

The first theorem generalizes a well-known result about the vector space R". 

Note 

A vector space V with a basis B containing n vectors is isomorphic to R" i.e., there 
exist a one-to-one linear transfonnation from V to R". 

Theorem 1 If a vector space Khas a basis B = {bj, ..., b„}, then any set in V containing 
more than n vectors must be linearly dependent. 

Theorem 2 If a vector space V has a basis of n vectors, then every basis of V must 
consist of exactly n vectors. 

Finite and infinite dimensional vector spaces 

If the vector space V is spanned or 
generated by a finite set, then V is said to be finite-dimensional, and the dimension of V, 
written as dim V, is the number of vectors in a basis for V. If V is not spanned by a finite 
set, then V is said to be infinite-dimensional. That is, if we are unable to find a finite set 
that can generate the whole vector space, then such a vector space is called infinite 
dimensional. 

Note 


(1) The dimension of the zero vector space {0} is defined to be zero. 

(2) Every finite dimensional vector space contains a basis. 


Example 1 The n dimensional set of real numbers R n , set of polynomials of order n 
P n , and set of matrices of order mxn M mn are all finite- dimensional vector spaces. 
However, the vector spaces F (-00,00), C (-00,00), and C m (-00,00) are infinite¬ 
dimensional. 

Example 2 

(a) Any pair of non-parallel vectors a, b in the xy-plane, which are necessarily linearly 
independent, can be regarded as a basis of the subspace R 2 . In particular the set of unit 
vectors (i, jj forms a basis for R 2 . Therefore, dim (R 2 ) = 2. 
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Any set of three non coplanar vectors {a , b, c} in ordinary (physical) space, which will be 
necessarily linearly independent, spans the space R 3 . Therefore any set of such vectors forms a 
basis for R 3 . In particular the set of unit vectors {i, j, k} forms a basis of R 3 . This basis is called 
standard basis forif^. Therefore dim (R 3 ) = 3. 

The set of vectors {ei, e 2 , .... e„} where 

d = (1, 0, 0, 0, ..., 0), 

e 2 = (0, 1, 0, 0, ..., 0), 
e 3 = (0, 0, 1, 0, ..., 0), 


e n = (0, 0, 0, 0, 1) 

is linearly independent. 

Moreover, any vector x = (xi, x 2 , .... x n ) in R" can be expressed as a linear combination of these 
vectors as 

x =xiej + x 2 e 2 + x 3 es +... + x„e„. 

Hence, the set {e 2 , e 2 , , e n j forms a basis for if". It is called the standard basis of if", therefore 

dim (R n ) = n. Any other set of n linearly independent vectors in if" will fonn a non-standard 
basis. 

(b) The set B = {1, x, x 2 , ... ,x n } forms a basis for the vector space P„ of polynomials of degree 
< n. It is called the standard basis with dim (P„) = n + 1. 

(c) The set of 2 x 2 matrices with real entries (elements) {ui, u 2 , u 3 , U4} where 


"1 o' 


"0 1 ] 


1 

0 

0 


"0 0" 


II 

a 


, U 3 = 


a 

11 


0 0 


0 0 


1 0 


0 


is a linearly independent and every 2x2 matrix with real entries can be expressed as their linear 
combination. Therefore, they fonn a basis for the vector space M 2X2 . This basis is called the 
standard basis for M 2X2 with dim (M 2X2 ) = 4. 

Note 


( 1 ) dim (R n ) = n { The standard basis has n vectors}. 

(2) dim (P„) = n + 1 { The standard basis has n+1 vectors}. 

(3) dim (M m x „) = mn { The standard basis has mn vectors.} 

Example 3 Let Wbe the subspace of the set of all (2 x 2) matrices defined by 

W = {A = : 2a - b + 3c + d = 0}. 

c d 

Determine the dimension of W. 

Solution The algebraic specification for W can be rewritten as d = -2a + b - 3c. 
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Now 


a b 
c d 

Substituting the value of d, it becomes 


a b 

c - 2 a + b - 3c 


This can be written as 



a 0 


"0 

b~ 


"0 

0 " 

A= 

0 -2a 

+ 

0 

b 

+ 

c 

-3c 



'1 

0 ' 

+ b 

"0 1' 


0 

0 

a 

0 

-2 

0 1 

+ c 

! -3 



= a Aj + bA 2 + cA 3 




"1 0 ' 


"0 r 


0 

0 

where 

Ai = 

0 -2 

,a 2 = 

0 i_ 

, and A 3 = 

! -3 


The matrix A is in W if and only if A = aAj + bA 2 + cA 3, so (A /, A 2 , A3} is a spanning set for 
W. Now, check if this set is a basis for W or not. We will see whether (A /, A 2 , A3} is linearly 
independent or not. {A lt A 2 , A3} is said to be linearly independent if 
aAj + bA 2 + cA 3 =0 =>a=b=c=0 i.e., 


1 0 


+ b 


0 1 


+ c 


0 0 


0 0 


L° - 2 _ 


Lo 

1 

J 

Li -3J 

L° 0 

a 0 

+ 

"0 , 

b~ 

+ 

"0 

0 ' 


1- 

O 

O 

1_ 

0 -2a 


0 , 

b _ 


c 

-3c} 


L° oj 

a 

b 

" 


"0 

O' 




c -2a + b 

-3c 


0 

0 





Equating the elements, we get 
a = 0 ,b = 0,c = 0 

This implies {Ai, A 2 , A3} is a linearly independent set that spans W. Hence, it’s the basis of W 
with dim( W)= 3. 



"3] 



Example 4 Let H = Span !vi, v 2 l, where v, = 

6 

and v 2 = 

0 


2 


1 


Then H is the plane 


studied in Example 10 of lecture 23. A basis for H is {vi, v 2 }, since V] and v 2 are not 
multiples and hence are linearly independent. Thus, dim H = 2 . 
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A coordinate system on a plane Hin R 3 


Example 5 Find the dimension of the subspace 

a-3b+6c 
5 a + 4 d 
b-2c-d 
5 d 


H = \ 


: a,b,c,d e R 


Solution The representative vector of H can be written as 


a - 3 b + 6c 


T 


~-3 


' 6 ' 


' 0 " 

5 a + 4 d 

= a 

5 

+b 

0 

+ c 

0 

+ d 

4 

b-2c-d 


0 


1 


-2 


-1 

5 d 


0 _ 


0 _ 


0 _ 


5 


Now, it is easy to see that H is the set of all linear combinations of the vectors 


V 


“-3" 


“6“ 


“O' 

5 


0 


0 


4 

0 

- V 2 = 

1 

- = 

-2 

- v, = 

-1 

0 


0 


0 


_ 5 _ 


Clearly, v, ^ 0 ,v 2 is not a multiple of V/, but v? is a multiple of v 2 . By the Spanning Set 
Theorem, we may discard v? and still have a set that spans H. Finally; V4 is not a linear 
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combination of Vi and V 2 . So {vi, V 2 , V 4 } is linearly independent and hence is a basis for 
H. Thus dim H = 3. 

Example 6 The subspaces of R 3 can be classified by various dimensions as shown in 
Fig. 1. 

O-dimensional subspaces 

The only O-dimensional subspace of if 3 is zero space. 

1- dimensional subspaces 

1 -dimensional subspaces include any subspace spanned by a single non-zero 
vector. Such subspaces are lines through the origin. 

2- dimensional subspaces 

Any subspace spanned by two linearly independent vectors. Such subspaces are 
planes through the origin. 

3- dimensional subspaces 

The only 3-dimensional subspace is if 3 itself. Any three linearly independent 
vectors in R 3 span all of R 3 , by the Invertible Matrix Theorem. 


*3 



3 dim 


*3 




Figure 1 - Sample subspaces of R 3 
Bases for Nul l and Col.l 

We already know how to find vectors that span the null 
space of a matrix A. The discussion in Lecture 21 pointed out that our method always 
produces a linearly independent set. Thus the method produces a basis for Nul A. 
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Example 7 Find a basis for the null space of A = 


-1 -1 


-1 0 1 

2 -3 1 


1 1-2 0-1 

0 0 111 


Solution The null space of A is the solution space of homogeneous system 


2xj + 2x 2 - x 3 


x 5 =0 


-Xj- x 2 + 2x 3 - 3x 4 + x 5 = 0 


x , - 2x 3 


-x 5 =0 
+ x 5 =0 


The most appropriate way to solve this system is to reduce its augmented matrix into 
reduced echelon form. 

[ 22-101 0 ] 


R 4 ~ R 2 , R 3 ~ /[ 


R ?i -2R t , R 3 - 3R 2 


R 3 -3R 2 


-U 


R 4 + R ] 
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1 1-2 0-1 
0 0 111 
0 0 0 1 0 

0 0 0 -3 0 


0 

0 

0 

0 


R 4 + 3i? 3 


1 1-2 0-1 
0 0 111 

0 0 0 1 0 

0 0 0 0 0 

1 10 2 1 
0 0 111 

0 0 0 1 0 

0 0 0 0 0 


0 

0 

0 

0 

0 

0 

0 

0 


+ 2 R^ 


r 2 -r 3 , r 1 


-2R 


3 


110 0 1 
0 0 10 1 
0 0 0 1 0 
0 0 0 0 0 


0 

0 

0 

0 


Thus, the reduced row echelon form of the augmented matrix is 


110 0 1 
0 0 10 1 
0 0 0 1 0 
0 0 0 0 0 


0 

0 

0 

0 


which corresponds to the system 

1 Xj +1 x 2 + 1 x 5 =0 

1 x 3 + 1 = 0 

lx 4 =0 


0 = 0 


No equation of this system has a fonn zero = nonzero. Therefore, the system is 
consistent. Since the number of unknowns is more than the number of equations, we will 
assign some arbitrary value to some variables. This will lead to infinite many solutions of 
the system. 
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x, 


- lx 2 -lx 5 


x 3 = - lx 5 
X, =0 


x 5 =t 


The general solution of the given system is 

X] = -S-t , X2 = S , Xs = -t, X4 = 0 , 
Therefore, the solution vector can be written as 


x 5 = t 


Xj 


-s-t 


-s 






-1 

X2 


s 


s 


0 


1 


0 

x 3 

= 

-t 

= 

0 

+ 

-t 

= s 

0 

+ t 

-1 

X4 


0 


0 


0 


0 


0 

_v 


t 


0 _ 


t 


_°\ 


1 


which shows that the vectors v 3 


"-ll 


“-f 

1 


0 

0 

and v 2 = 

-1 

0 


0 

0 


1 


span the solution space .Since they 


are also linearly independent, {vi,V 2 } is a basis for NulA 

The next two examples describe a simple algorithm for finding a basis for the column 
space. 


Example 8 Find a basis for Col B, where B = [b : b 2 . 


b s ] = 


1 4 0 2 0 

0 0 1-10 
0 0 0 0 1 

0 0 0 0 0 

Solution Each non-pivot column of B is a linear combination of the pivot columns. In 
fact, b 2 = 4bi and b 4 = 2b i - b i. By the Spanning Set Theorem, we may discard b 2 and 
b 4 and {bi, b 3 , b 5 } will still span Col B. Let 


S = {b 2 , b 3 , b 5 } = 



~r 


"0“ 


"0“ 



0 


1 


0 



0 


0 


1 



0 


0 


0 



Since bi^O and no vector in S is a linear combination of the vectors that precede it, S is 
linearly independent. Thus S is a basis for Col B. 

What about a matrix A that is not in reduced echelon form? Recall that any 
linear dependence relationship among the columns of A can be expressed in the form Ax 


@Virtual University Of Pakistan 


330 




= 0, where x is a column of weights. (If some columns are not involved in a particular 
dependence relation, then their weights are zero.) When A is row reduced to a matrix B , 
the columns of B are often totally different from the columns of A. However, the 
equations Ax = 0 and Bx = 0 have exactly the same set of solutions. That is, the columns 
of A have exactly the same linear dependence relationships as the columns of B. 

Elementary row operations on a matrix do not affect the linear dependence relations 
among the columns of the matrix. 

Example 9 It can be shown that the matrix 

1 4 0 2 -1" 

3 12 1 5 5 

2 8 13 2 

5 20 2 8 8_ 

is row equivalent to the matrix B in Example 8. Find a basis for Col A. 

Solution In Example 8, we have seen that b 2 = 4b t and b 4 = 2b 4 - b 3 

so we can expect that a 2 = 4a, and a 4 = 2a 7 - a 3 . This is indeed the case. 
Thus, we may discard a 2 and a 4 while selecting a minimal spanning set for Col A. In fact, 
{ai, a 3 , a 5 } must be linearly independent because any linear dependence relationship 
among a ly a 3 , as would imply a linear dependence relationship among bi, b 3 , bs . But we 
know that {bi, b 3 , bs} is a linearly independent set. Thus {aj, U 3 , as} is a basis for Col A 
The columns we have used for this basis are the pivot columns of A. 

Examples 8 and 9 illustrate the following useful fact. 

Theorem 3 The pivot columns of a matrix A form a basis for Col A. 

Proof The general proof uses the arguments discussed above. Let B be the reduced 
echelon fonn of A. The set of pivot columns of B is linearly independent, for no vector in 
the set is a linear combination of the vectors that precede it. Since A is row equivalent to 
B , the pivot columns of A are linearly independent too, because any linear dependence 
relation among the columns of A corresponds to a linear dependence relation among the 
columns of B. For this same reason, every non-pivot column of A is a linear combination 
of the pivot columns of A. Thus the non-pivot columns of A may be discarded from the 
spanning set for Col A, by the Spanning Set Theorem. This leaves the pivot columns of A 
as a basis for Col A 

Note Be careful to use pivot columns of A itself for the basis of Col A. The columns of 
an echelon form B are often not in the column space of A. For instance, the columns of 
the B in Example 8 all have zeros in their last entries, so they cannot span the column 
space of the A in Example 9. 
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1 


-2 

Example 10 Let v, = 

-2 

and v 2 = 

7 


3 


-9 


Determine if {vi, V 2 I is a basis for if 2 . Is {Vi, 


y 2 } a basis for if 2 ? 
Solution Let A = [v] 


v 2 ] ■ Row operations show that A = 

1 

<N 

h <N 

1 _ 

_ 

“l -2 

0 3 


3 -9 


0 0 


. Not every 


row of A contains a pivot position. So the columns of A do not span if 2 , by Theorem 4 in 
Lecture 6. Hence {vi, V 2 } is not a basis for R 3 . Since Vj and V 2 are not in if 2 , they cannot 
possibly be a basis for if 2 . However, since Vi and V 2 are obviously linearly independent, 
they are a basis for a subspace of if 2 , namely, Span {v h v 2 }. 



'll 


6 


' 21 


~-4 

Example 11 Let v, = 

-3 

>*2 = 

2 

>*3 = 

-2 

>*4 = 

-8 


4 


-1 


3 


9 


Find a basis for the subspace 


W spanned by {vi, V2, V3, V4}. 

Solution Let A be the matrix whose column space is the space spanned by {vi, V 2 , V 3 , 

V4}, 


A = 


1 

-3 

4 


6 

2 

-1 


2 

-2 

3 


-4 

-8 

9 


Reduce the matrix A into its echelon form in order to find its pivot columns. 



" 1 

6 

2 ■ 

4 

A = 

-3 

2 

-2 ■ 

-8 


4 

-1 

3 

9 


'1 

6 

2 

-4 

~ 

0 

20 

4 ■ 

-20 


0 

-25 

-5 

25 


'1 

6 2 

-4] 


~ 

0 

5 1 

-5 

h 


0 

0 0 

0 



by R 2 + 3if,, if 3 - 4ifj 


1 

4 


The first two columns of A are the pivot columns and hence form a basis of Col A = W. 
Hence {vi, v 2 } is a basis for W. 

Note that the reduced echelon fonn of A is not needed in order to locate the pivot 
columns. 
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Procedure 

Basis and Linear Combinations 

Given a set of vectors S = {vi, V2, ...,Vk} in R", the following procedure produces a subset 
of these vectors that form a basis for span ( S) and expresses those vectors of S that are 
not in the basis as linear combinations of the basis vector. 

Stepl: Form the matrix,4 having vi, V2,..., v* as its column vectors. 

Step2: Reduce the matrix A to its reduced row echelon form if, and let 
wi, n>2,..., Wk be the column vectors of R. 

Step3: Identify the columns that contain the leading entries i.e., l’s in R. The 
corresponding column vectors of A are the basis vectors for span ( S ). 

Step4: Express each column vector of R that does not contain a leading entry as 

a linear combination of preceding column vector that do contain leading entries 
(we will be able to do this by inspection). This yields a set of dependency 
equations involving the column vectors of R. The corresponding equations for the 
column vectors of A express the vectors which are not in the basis as linear 
combinations of basis vectors. 


Example 12 Basis and Linear Combinations 

(a) Find a subset of the vectors Vi = (1, -2, 0, 3), V 2 = (2, -4, 0, 6 ), v? = (-1, 1, 2, 0) and 
V 4 = (0, -1, 2, 3) that form a basis for the space spanned by these vectors. 

(b) Express each vector not in the basis as a linear combination of the basis vectors. 
Solution (a) We begin by constructing a matrix that has V/, V 2 , V 3 , v 4 as its column 
vectors 

"1 2 - 10 ' 

-2 -4 1 -1 

0 0 2 2 

3 6 0 3 

t t t t (A) 


v , V, v 4 

Finding a basis for column space of this matrix can solve the first part of our problem. 
Transforming Matrix to Reduced Row Echelon Fonn: 


1 

-2 

0 

3 

1 

0 

0 

0 


2 

-4 

0 

6 

2 

0 

0 

0 


-1 

1 

2 

0 

-1 

-1 

2 

3 


0 

-1 

2 

3_ 

0 

-1 

2 

3 


2 Rj + R 1 
-3R, + R 4 
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1 

0 

0 

0 

1 

0 

0 

0 

1 

0 

0 

0 


2 

0 

0 

0 

2 

0 

0 

0 

2 

0 

0 

0 


-1 

1 

2 

3 

-1 

1 

0 

0 

0 

1 

0 

0 


0 

1 

2 

3 

0 

1 

0 

0 

1 

1 

0 

0 


- 1*2 

-2R 2 +R 3 
-3R : + R, 


R 2 + R i 


Labeling the column vectors of the resulting matrix as Wi, tv 2 , tv 3 and tv 4 yields 

"1 2 0 ll 

0 0 11 
0 0 0 0 

0 0 0 0 


(B) 


t t t t 


tv, tv 2 tv 3 w 4 

The leading entries occur in column 1 and 3 so {wi, tv 3} is a basis for the column space 
of (B) and consequently {vi, v 3 } is the basis for column space of (A). 

(b) We shall start by expressing n > 2 and tv 4 as linear combinations of the basis vector \v 1 
and tv3. The simplest way of doing this is to express tv 2 and tv4 in tenn of basis vectors 
with smaller subscripts. Thus we shall express tv 2 as a linear combination of tv 1, and we 
shall express tv4 as a linear combination of tvj and tv3. By inspection of (B), these linear 
combinations are tv 2 = 2 tv t and tv4 = tv t + tv3. We call them the dependency equations. 
The corresponding relationship of (A) are v.? = 2vi and V 5 = v? + V3. 


Example 13 Basis and Linear Combinations 

(a) Find a subset of the vectors Vi = (1,-1, 5, 2), v 2 = (-2, 3, 1, 0), v 3 = (4, -5, 9, 4), 

V 4 = (0, 4, 2, -3) and v$ = (-7, 18, 2, - 8 ) that form a basis for the space spanned by these 
vectors. 

(b) Express each vector not in the basis as a linear combination of the basis vectors 
Solution (a) We begin by constructing a matrix that has Vj, v 2 , ... , V 5 as its column 
vectors 
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1 -2 4 0 -7 

-13-5 4 18 

5 19 2 2 

(A) 

2 0 4 -3 -8 v ’ 

V; v 2 v, v 4 v 5 

Finding a basis for column space of this matrix can solve the first part of our problem. 
Transforming Matrix to Reduced Row Echelon Fonn: 


" 1 

-2 

4 

0 

-7 " 


-1 

3 

-5 

4 

18 


5 

1 

9 

2 

2 


2 

0 

4 

-3 

-8 


" 1 

-2 

4 

0 

-7 “ 

R , + R 7 

0 

1 

-1 

4 

11 

1 Z 

-5R, +R, 

0 

11 

-11 

2 

37 

1 j 

-2R, + R, 

0 

4 

-4 

-3 

6 

1 4 

" 1 

-2 

4 

0 

-7 


0 

1 

-1 

4 

11 

-11R 2 +R 

0 

0 

0 

-42 

-84 

-4R 2 + R 4 

0 

0 

0 

-19 

-38 


" 1 

-2 

4 

0 

-7 


0 

1 

-1 

4 

11 

(-1/42)R 3 

0 

0 

0 

1 

2 

0 

0 

0 

-19 

-38 


" 1 

-2 

4 

0 

-7 “ 


0 

1 

-1 

4 

11 

19R 3 + R. 

0 

0 

0 

1 

2 


0 

0 

0 

0 

0 


" 1 

-2 

4 

0 

-7 “ 


0 

1 

-1 

0 

3 

(-4)R 3 +R 2 

0 

0 

0 

1 

2 

0 

0 

0 

0 

0 


" 1 

0 

2 

0 

-T 


0 

1 

-1 

0 

3 

2Rj + R 

0 

0 

0 

1 

2 

Z 1 

0 

0 

0 

0 

0 
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Denoting the column vectors of the resulting matrix by Wi, tv 2 , w > 3 , W 4 , and Ws yields 
1 0 2 0 -1 

01 -103 

0 0 0 1 2 

0 0 0 0 0 

^ ^ ^ ^ ^ 

W, IV 2 w 3 \v 4 w 5 

The leading entries occur in columns 1,2 and 4 so that {wj, W 2 , n’ 4 } is a basis for the 
column space of (B) and consequently {vi, V 2 , V 4 } is the basis for column space of (A). 

(b) We shall start by expressing W 3 and W 5 as linear combinations of the basis vector n>i, 
n> 2 , W 4 . The simplest way of doing this is to express W 3 and W 5 in tenn of basis vectors 
with smaller subscripts. Thus we shall express W 3 as a linear combination of w\ and W 2 , 
and we shall express wjasa linear combination of Wi, W 2 , and W 4 , By inspection of (B), 
these linear combination are W 3 = 2 n>i - w 2 and w 5 = -Wi + 3w 2 + 2 n> 4 . 

The corresponding relationship of (A) are v.? = 2vi -V 2 and V 5 = -Vi + 3v 2 + 2v4. 



Example 14 Basis and Linear Combinations 

(a) Find a subset of the vectors V 7 = (1, -2, 0, 3), v 2 = (2, -5, -3, 6), V 3 = (0, 1, 3, 0), 

V 4 = (2, -1, 4, -7) and V 5 = (5 , - 8 , 1, 2) that form a basis for the space spanned by these 
vectors. 

(b) Express each vector not in the basis as a linear combination of the basis vectors. 
Solution (a) We begin by constructing a matrix that has V 7 , V 2 , ... , V 5 as its column 
vectors 

" 1 2 0 2 5“ 

-2 -5 1 -1 -8 

0 -3 3 4 1 

_ 3 6 0 -7 2 _ 

t t t t t (A) 


^7 v, v 4 v 5 

Finding a basis for column space of this matrix can solve the first part of our problem. 
Reducing the matrix to reduced-row echelon form and denoting the column vectors of the 
resulting matrix by Wi, n>2, W3, W4, and \v 4 yields 
“l 0 2 0 1 “ 

0 1 -101 
0 0 0 1 1 

0 0 0 0 oj (B) 

^ ^ ^ ^ ^ 


w, w 2 w 3 w 4 w 5 

The leading entries occur in columns 1, 2 and 4 so {wi, W 2 , W 4 } is a basis for the column 
space of (B) and consequently {vi, V2, V4} is the basis for column space of (A). 
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(b) Dependency equations are Wj = 2 wi - tv2 and W5 = wi + w2 + W4 
The corresponding relationship of (A) are V3 = 2 vi -V 2 and Vs = Vi + V2 + V4 

Subspaces of a Finite-Dimensional Space The next theorem is a natural counterpart 
to the Spanning Set Theorem. 

Theorem 5 Let H be a subspace of a finite-dimensional vector space V. Any linearly 
independent set in H can be expanded, if necessary, to a basis for H. Also, H is finite - 
dimensional and dim/T < dimK . 


When the dimension of a vector space or subspace is known, the search for a basis is 
simplified by the next theorem. It says that if a set has the right number of elements, then 
one has only to show either that the set is linearly independent or that it spans the space. 
The theorem is of critical importance in numerous applied problems (involving 
differential equations or difference equations, for example) where linear independence is 
much easier to verify than spanning. 

Theorem 5 (The Basis Theorem) Let Fbe a /^-dimensional vector space, p> 1. Any 
linearly independent set of exactly p elements in V is automatically a basis for V. Any set 
of exactly p elements that spans V is automatically a basis for V. 

The Dimensions of Nul A and Col A Since the pivot columns of a matrix A form a 
basis for Col A, we know the dimension of Col A as soon as we know the pivot columns. 
The dimension of Nul A might seem to require more work, since finding a basis for Nul 
A usually takes more time than a basis for Col A. Yet, there is a shortcut. 

Let A be an m x n matrix, and suppose that the equation Ax = 0 has k free variables. 
From lecture 21, we know that the standard method of finding a spanning set for Nul A 
will produce exactly k linearly independent vectors say, uj, ... , Uk, one for each free 
variable. So {uj, ... , u k / is a basis for Nul A, and the number of free variables determines 
the size of the basis. Let us summarize these facts for future reference. 


The dimension of Nul A is the number of free variables in the equation Ax = 0, and the 
dimension of Col A is the number of pivot columns in A. 


Example 15 


Find the dimensions of the null space and column space of 



6 -1 1 

-2 2 3 

-4 5 8 


-7 

-1 

-4 


Solution Row reduce the augmented matrix [A 0] to echelon form and obtain 
"1 -2 2 3 -1 0" 

0 0 12-20 
0 0 0 0 0 0 


@Virtual University Of Pakistan 


337 



24-Dimension of a Vector Space 


VU 


Writing it in equations form, we get 

x x - 2x 2 + 2x 3 + 3x 4 - x 5 = 0 

x 3 + 2x 4 - 2x 5 = 0 

Since the number of unknowns is more than the number of equations, we will introduce 
free variables here (say) X 2 , X 4 and xj. Hence the dimension of Nul A is 3. Also dim Col 
A is 2 because A has two pivot columns. 

Example 16 Decide whether each statement is true or false, and give a reason for each 
answer. Here V is a non-zero finite-dimensional vector space. 

1. If dim V = p and if A is a linearly dependent subset of V, then A contains more than 
p vectors. 

2. If A spans V and if T is a subset of V that contains more vectors than A, then T is 
linearly dependent. 

Solution 

1. False. Consider the set { 0 }. 

2. True. By the Spanning Set Theorem, A contains a basis for V; call that basis A'. 
Then T will contain more vectors than A'. By Theorem 1, T is linearly dependent. 
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Exercises 

For each subspace in exercises 1-6, (a) find a basis and (b) state the dimension. 





2c 


s-2t 


2. < 

a -6 

: a, b,c in R 

s + t 

:s,tinR 

6-3c 


\_3t 








a + 2b 




a-4b-2c 



3a + 66-c 



2a + 5b- 4c 



6 a-2b- 2c 


3. < 

-a + 2c 

:a,b,cinR 

4. < 

-9a + 56 + 3c 

:a,b,cinR 


-3a+ 76 +6c 



-3a + b + c 



5. {(a, b, c): a - 3b + c = 0, b - 2c = 0, 2b - c = 0} 

6 {(a, b, c, d): a - 3b + c = 0} 

7. Find the dimension of the subspace //of R 2 spanned by 


"2 " 

-4 

"-3" 

-5 ' 

’|_10_' 

L 6 _ 


8. Find the dimension of the subspace spanned by the given vectors. 



Determine the dimensions of Nul^ and Col A for the matrices shown in exercises 9 to 
12 . 




1 

-6 

9 

0 

-2] 


'1 

3 

-4 

2 

-1 

6 " 


0 

1 

2 

-4 

5 


0 

0 

1 

-3 

7 

0 

9. A = 







10. A = 








0 

0 

0 

5 

1 


0 

0 

0 

1 

4 

-3 


0 

0 

0 

0 

0 


0 

0 

0 

0 

0 

0 







'1 

-1 

o' 





'1 

0 

9 

5 “ 








11. A = 







12. A = 

0 

4 

7 





0 

0 

1 

-4 














0 

0 

5 
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13. The first four Hermite polynomials are 1, 2 1 , -2 + 41 2 , and -12 1 + 8 1 3 . These 
polynomials arise naturally in the study of certain important differential equations in 
mathematical physics. Show that the first four Hermite polynomials form a basis of Pj. 

14. Let B be the basis of P 3 consisting of the Hennite polynomials in exercise 13, and let 
p (t) = 7 - 12 t - 8 t 2 + 12 ?. Find the coordinate vector of p relative to B. 

15. Extend the following vectors to a basis for R 5 : 
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Lecture 25 


Rank 


With the help of vector space concepts, for a matrix several interesting and useful 
relationships in matrix rows and columns have been discussed. 

For instance, imagine placing 2000 random numbers into a 40 x 50 matrix A and then 
determining both the maximum number of linearly independent columns in A and the 
maximum number of linearly independent columns in A T (rows in A). Remarkably, the 
two numbers are the same. Their common value is called the rank of the matrix. To 
explain why, we need to examine the subspace spanned by the subspace spanned by the 
rows of A. 


The Row Space If A is an m x n matrix, each row of A has n entries and thus can be 
identified with a vector in if". The set of all linear combinations of the row vectors is 
called the row space of ,4 and is denoted by Row A Each row has n entries, so Row ,4 is 
a subspace of if". Since the rows of A are identified with the columns of A r , we could 
also write Col A T in place of Row A. 


Example 1 


Let A = 


-2 

1 

3 

1 


-5 

3 

11 

7 


8 

-5 

-19 

-13 


0 -17 
1 5 

7 1 

5 -3 


r, =(-2,-5,8,0,-17) 
r 2 = (1,3,-5,1,5) 
r 3 = (3,11,-19,7,1) 
r 4 = (1,7,-13,5,-3) 


The row space of A is the subspace of if 5 spanned by {r 2 , r 2 , r 3 , That is, Row A = 
Span {r ly r 2 , r 3 , r 4 }. Naturally, we write row vectors horizontally; however, they could 
also be written as column vectors 

Example Let 


A = 


2 

3 


1 0 
-1 4 


and 


r, =(2,1,0) 

r 2 = (3,-1,4) 


That is Row A=Span {r ly r 2 }. 

We could use the Spanning Set Theorem to shrink the spanning set to a 

basis. 

Some times row operation on a matrix will not give us the required infonnation but row 
reducing certainly worthwhile, as the next theorem shows 

Theorem 1 If two matrices A and B are row equivalent, then their row spaces are the 
same. If B is in echelon form, the nonzero rows of B form a basis for the row space of A 
as well as B. 

Theorem 2 If A and B are row equivalent matrices, then 
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(a) A given set of column vectors of A is linearly independent if and only if the 
corresponding column vectors of B are linearly independent. 

(b) A given set of column vector of A forms a basis for the column space of A if and only 
if the corresponding column vector of B forms a basis for the column space of B. 


Example 2 (Bases for Row and Column Spaces) 


Find the bases for the row and column spaces of A = 


1 

2 

2 

-1 


-3 4 -2 5 4 

- 69-182 
- 69-197 
3-4 2-5 -4 


Solution We can find a basis for the row space of A by finding a basis for the row 
space of any row-echelon form of A. 



" 1 

-3 

4 


-2 


5 

4 




2 

-6 

9 


-1 


8 

2 



Now 












2 

-6 

9 


-1 


9 

7 




-1 

3 

-4 


2 


-5 

-4 



' 1 

-3 

4 


-2 


5 

4 

" 












-2 R, + R, 


0 

0 

1 


3 


-2 

-6 

1 Z 











-2 R, + R, 


0 

0 

1 


3 


-1 

-1 

1 j 











R, + R, 


0 

0 

0 


0 


0 

0 


1 4 


' 1 

-3 

4 


-2 


5 

4 




0 

0 

1 


3 


-2 

-6 












- IRj + R, 


0 

0 

0 


0 


1 

5 


Z J 


0 

0 

0 


0 


0 

0 





'1 

-3 

4 

-2 

5 

4" 





0 

0 

1 

3 

-2 

-6 



Row-echelon form of A 

: R = 











0 

0 

0 

0 

1 

5 





0 

0 

0 

0 

0 

0 




Here Theorem 1 implies that that the non zero rows are the basis vectors of the matrix. 
So these bases vectors are 


*7 = [1 -3 4 -2 5 4] 
r 2 =[ 0 0 1 3 -2 -6] 
r,=[0 0 0 0 1 5] 

A and R may have different column spaces, we cannot find a basis for the column space 
of A directly from the column vectors of R. however, it follows from the theorem (2b) if 
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we can find a set of column vectors of R that forms a basis for the column space of if, 
then the corresponding column vectors of A will fonn a basis for the column space of A. 

The first, third, and fifth columns of if contains the leading l’s of the row vectors, so 

m m m 



_°J L°J L°_ 


form a basis for the column space of if, thus the corresponding column vectors of A 
'll [4] [5 

2 9 8 

namely, c, = c } = c = 

2 9 9 

_-iJ [- 4 ] |_-5 

form a basis for the column space of A 


Example 


The matrix 


"1 -2 5 0 3 

0 13 0 0 

R = 

0 0 0 1 0 

0 0 0 0 0 

is in row-echelon form. 

The vectors 

r 7 = [l -2 5 0 3 ] 

r 2 =[0 1 3 0 0 ] 

r 3 = [0 0 0 1 0 ] 

form a basis for the row space of R, and the vectors 


'll [-2] [0" 

0 1 0 



form a basis for the column space of R. 
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Example 3 (Basis for a Vector Space using Row Operation) 

Find bases for the space spanned by the vectors 

vj= (1,-2,0,0,3) v 2 = (2,-5,-3, -2,6) 

v 3 = (0,5,15,10,0) v 4 = (2,6,18,8,6) 

Solution The space spanned by these vectors is the row space of the matrix 

1 -2 0 0 3 

2 -5 -3 -2 6 

0 5 15 10 0 

2 6 18 8 6 


Transforming Matrix to Row Echelon Form: 

1-200 
2 -5 -3 -2 

0 5 15 10 

2 6 18 8 

1-200 
0 13 2 

0 5 15 10 

0 10 18 8 

1-200 
0 13 2 

0 0 0 0 

0 0 -12 -12 

1-200 
0 13 2 

0 0 -12 -12 

0 0 0 0 

1-200 
0 13 2 

0 0 11 

0 0 0 0 


3 

6 

0 

6 

Q (-2)R,+R; 

0 (-2)Rt+R 4 

(-1)R , 

0 2 

3 " 

0 (~5)R 2 +R 3 
0 (-10)R 2 + R 4 
0 



3 

0 

Q (-1/12)R 3 
0 
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1 -2 0 0 3 
0 13 2 0 

Therefore, R — 

0 0 110 
0 0 0 0 0 
The non-zero row vectors in this matrix are 

Wj = (1,-2,0,0,3), w 2 = (0,1,3,2,0), w 3 =(0,0,1,1,0) 

These vectors form a basis for the row space and consequently form a basis for the 
subspace of R 5 spanned by Vi, V 2 , V 3 . 

Example 4 (Basis for the Row Space of a Matrix) 

[1 -2 0 0 3" 

2 -5 -3 -2 6 

Find a basis for the row space of A = consisting entirely of row 

0 5 15 10 0 

2 6 18 8 6 

vectors from A. 

Solution We find A T ; then we will use the method of example (2) to find a basis for 
the column space of A 7 ; and then we will transpose again to convert column vectors back 
to row vectors. Transposing A yields 
[12 0 2 

-2-5 5 6 

A t = 0 -3 15 18 

0 -2 10 8 
3 6 0 6 

Transforming Matrix to Row Echelon Form: 

12 0 2 

-2-5 5 6 

0 -3 15 18 

0 -2 10 8 

3 6 0 6 

12 0 2 
0 -1 5 10 

0 -3 15 18 

0 -2 10 8 

0 0 0 0 


2 Rj+R 2 

(-3)R 1 +R 5 
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Now 


1 

0 

0 

0 

0 

1 

0 

0 

0 

0 

1 

0 

0 

0 

0 

1 

0 

0 

0 

0 


R 


2 

1 

-3 

-2 

0 

2 

1 

0 

0 

0 

2 

1 

0 

0 

0 

2 

1 

0 

0 

0 

1 2 


0 

-5 

15 

10 

0 

0 

-5 

0 

0 

0 

0 

-5 

0 

0 

0 

0 

-5 

0 

0 

0 


2 

-10 

18 

8 

0 

2 

-10 

-12 

-12 

0 

2 

-10 

1 

-12 

0 

2 

-10 

1 

0 

0 


(-m 2 


m 2 +R 3 

(2)R 2 +R 4 


(-1/12) R s 


12 R 3 +R 4 


0 


0 1 -5 -10 

0 0 0 1 

0 0 0 0 

0 0 0 0 


The first, second and fourth columns contain the leading l’s, so the corresponding 
column vectors in A r form a basis for the column space ofA T ; these are 


“ 1 “ 


~2" 


"2" 

-2 


-5 


6 

0 

> C 2 

-3 

andc, = 

18 

0 


-2 


8 

_3 _ 


6 


6 


Transposing again and adjusting the notation appropriately yields the basis vectors 
r 7 =[l -2 0 0 3],r 2 =[2 -5 -3 -2 6\cind r 4 = [2 6 18 8 6] 
for the row space of A 
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The following example shows how one sequence of row operations on A leads to bases 
for the three spaces: Row A, Col A, and Nul A. 


Example 5 Find bases for the row space, the column space and the null space of the 
matrix 


-2 -5 8 

1 3 -5 

3 11 -19 

1 7 -13 


0 -17 
1 5 

7 1 

5 -3 


Solution To find bases for the row space and the column space, row reduce A to an 


echelon form: A ~ B = 


1 3 
0 1 
0 0 
0 0 


-5 

-2 

0 

0 


1 5 

2 -7 
-4 20 
0 0 


By Theorem (1), the first three rows of B form a basis for the row space of A (as well as 
the row space of B ). Thus Basis for Row .4: 

{(1, 3, -5, 1, 5), (0, 1, -2, 2, -7), (0, 0, 0, -4, 20)} 

For the column space, observe from B that the pivots are in columns 1, 2 and 4. Hence 
columns 1, 2 and 4 of A (not B ) fonn a basis for Col .4: 



Any echelon form of A provides (in its nonzero rows) a basis for Row A and also 
identifies the pivot columns of A for Col A. However, for Nul A, we need the reduced 
echelon form. Further row operations on B yield 


A ~ B ~ C = 


1 0 
0 1 
0 0 
0 0 


1 

-2 

0 

0 


0 1 
0 3 

1 -5 
0 0 


The equation Ax = 0 is equivalent to Cx = 0, that is, 


Xj + 


■ 2x, 


+ x 5 = 0 
+ 3x 5 = 0 
- 5x 5 = 0 
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So x/ = -X 3 - xs, x? = 2 x 3 ~ 3 x 5 , %4 = 5xs, with X 3 and xj free variables. The usual 
calculations (discussed in lecture 21) show that 


Basis for Nul .4 :< 


-1 

2 

1 

0 

0 


-1 

-3 

0 > 
5 
1 


Observe that, unlike the bases for Col A, the bases for Row A and Nul A have no simple 
connection with the entries in A itself 

Note 

1. Although the first three rows of B in Example (5) are linearly independent, it is wrong 
to conclude that the first three rows of A are linearly independent. (In fact, the third 
row of A is 2 times the first row plus 7 times the second row). 

2. Row operations do not preserve the linear dependence relations among the rows of a 
matrix. 

Definition The rank of A is the dimension of the column space of A. 

Since Row A is the same as Col A 7 , the dimension of the row space of A is the rank of 
A 7 . The dimension of the null space is sometimes called the nullity of ,4. 

Theorem 3 (The Rank Theorem) The dimensions of the column space and the row 
space of an m x n matrix .4 are equal. This common dimension, the rank of A, also equals 
the number of pivot positions in A and satisfies the equation 
ra nk A + dim Nul A = n 


Example 6 

(a) If A is a 7x9 matrix with a two - dimensional null space, what is the rank of A? 

(b) . Could a 6x9 matrix have a two - dimensional null space? 

Solution 

(a) Since A has 9 columns, (rank A) +2 = 9 and hence rank A = 7. 

(b) No, If a 6x9 matrix, call it B, had a two - dimensional null space, it would have to 
have rank 7, by the Rank Theorem. But the columns of B are vectors in R 6 and so the 
dimension of Col B cannot exceed 6 ; that is, rank B cannot exceed 6 . 

The next example provides a nice way to visualize the subspaces we have been studying. 
Later on, we will learn that Row A and Nul A have only the zero vector in common and 
are actually “perpendicular” to each other. The same fact will apply to Row A 7 (= Col A) 
and Nul A 7 . So the figure in Example (7) creates a good mental image for the general 
case. 
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Example 7 


Let A = 


3 0 

3 0 

4 0 


-1 

-1 


. It is readily checked that Nul A is the X 2 


5 


axis, Row 


A is the X 1 X 3 - plane, Col A is the plane whose equation is xj - xj = 0 and Nul A T is the 
set of all multiples of (1, -1, 0). Figure 1 shows Nul A and Row A in the domain of the 
linear transformation x —> Ax; the range of this mapping, Col A, is shown in a separate 
copy of R 3 , along with Nul A T . 



Figure 1 - Subspaces associated with a matrix ,4 


Applications to Systems of Equations 

The Rank Theorem is a powerful tool for processing information about systems of 
linear equations. The next example simulates the way a real-life problem using linear 
equations might be stated, without explicit mention of linear algebra terms such as 
matrix, subspace and dimension. 

Example 8 A scientist has found two solutions to a homogeneous system of 40 
equations in 42 variables. The two solutions are not multiples and all other solutions can 
be constructed by adding together appropriate multiples of these two solutions. Can the 
scientist be certain that an associated non-homogeneous system (with the same 
coefficients) has a solution? 

Solution Yes. Let A be the 40x42 coefficient matrix of the system. The given 
information implies that the two solutions are linearly independent and span Nul A. So 
dim Nul A = 2. By the Rank Theorem, dim Col A = 42 - 2 = 40. Since R 40 is the only 
subspace of if 40 whose dimension is 40, Col A must be all of R 40 . This means that every 
non-homogeneous equation Ax = b has a solution. 
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Example 9 


Find the rank and nullity of the matrix A 


-1 2 

3 -7 

2 -5 

4 -9 


Verify that values obtained verify the dimension theorem. 


Solution 


-1 

3 
2 

4 
1 

3 
2 

4 
1 
0 
0 
0 
1 
0 
0 
0 
1 
0 
0 
0 
1 
0 
0 
0 


2 

-7 

-5 

-9 

-2 

-7 

-5 

-9 

-2 

-1 

-1 

-1 

-2 

1 

-1 

-1 

-2 

1 

0 

0 

0 

1 

0 

0 


0 4 

2 0 

2 4 

2 -4 

0 -4 

2 0 

2 4 

2 -4 

0 -4 

2 12 

2 12 

2 12 

0 -4 

-2 -12 
2 12 

2 12 

0 -4 

-2 -12 
0 0 

0 0 

-4 -28 

-2 -12 
0 0 

0 0 


5 

-3 

1 

4 

6 

1 

-4 

7 

-5 

3 ' 

1 

4 

6 

1 

-4 

7 

-5 

3 

16 

-5 

16 

-5 

16 

-5 

-5 

3 

-16 

5 

16 

-5 

16 

-5 

-5 

3 " 

-16 

5 

0 

0 

0 

0 

-37 

13 

-16 

5 

0 

0 

0 

0 


(-VR, 


(-3)R j +R 2 
(~2)R] + R 3 
(- 4 )R , +R 4 


(-V R 2 


R 2 +R 3 
R 2 +R 4 


2 R 2 + R i 


The reduced row-echelon form of A is 

" 1 0 -4 -28 -37 13 

0 1 -2 -12 -16 5 

0 0 0 0 0 0 

0 0 0 0 0 0 


0 

2 

2 

2 


4 5-3 

0 1 4 

4 6 1 

-4 -4 7 


( 1 ) 


©Virtual University Of Pakistan 


350 





25-Rank 


VU 


The corresponding system of equations will be 

x, - 4x 3 - 28x 4 - 37x 5 + 13x 6 = 0 

x 2 - 2x 3 - 12x 4 - 16x 5 + 5x 6 = 0 
or, on solving for the leading variables, 

Xj = 4x 3 - 28x 4 + 3 7x 5 - 13x 6 

x 2 = 2x 3 + 12x 4 + 16x 5 - 5x 6 

it follows that the general solution of the system is 
Xj = 4r + 28s + 37t - 13u 

x 2 = 2r + 12s + 16t - 5u 
x 3 =r 
x 4 =s 
x 5 =t 

X 6 = u 

~Xj~\ \4~] [25] [37] [-13 

x 2 2 12 16 -5 

x 3 1 0 0 0 

or equivalently, =r +5 +t +u 

x 4 0 1 0 0 

x 5 0 0 1 0 

_x 6 _ 0 0 0 1 

The four vectors on the right side of (3) form a basis for the solution space, so 

"-1 2 0 4 5 -3] 

3 -7 2 0 1 4 

nullity (A) =4. The matrix A = has 6 columns, 

2 -5 2 4 6 1 

4-92-4-47 

so rank (A) + nullity^ =2 + 4 = 6 = n 


( 2 ) 
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Example 10 Find the rank and nullity of the matrix; then verify that the values 


obtained satisfy the dimension theorem A = 


1 

0 

2 

3 

-2 


-3 

3 

-3 

-6 

9 


2 

6 

-2 

0 

2 


2 1 
0 -3 

4 4 

6 5 

-4 -5 


Solution Transforming Matrix to the Reduced Row Echelon Form: 


1 

-3 

2 

2 

1 



0 

3 

6 

0 

-3 



2 

-3 

-2 

4 

4 



3 

-6 

0 

6 

5 



-2 

9 

2 

-4 

-5 



1 

-3 

2 

2 

1 



0 

3 

6 

0 

-3 

(-2)R, + R 3 

0 

3 

-6 

0 

2 

(~3)R, + R 4 

0 

3 

-6 

0 

2 

2R i +R 5 

0 

3 

6 

0 

-3 



1 

-3 

2 

2 

1 



0 

1 

2 

0 

-1 



0 

3 

-6 

0 

2 

(1/3) R 2 

0 

3 

-6 

0 

2 



0 

3 

6 

0 

-3 



1 

-3 

2 

2 

1 



0 

1 

2 

0 

-1 

(~3)R 2 +R 3 

0 

0 

-12 

0 

5 

(-3)R 2 +R 4 

0 

0 

-12 

0 

5 

(-3)R 2 + R 5 

0 

0 

0 

0 

0 



1 

-3 

2 

2 

1 

" 


0 

1 

2 

0 

-1 



0 

0 

1 

0 

-5/12 

(~1/12)R 3 

0 

0 

-12 

0 

5 



0 

0 

0 

0 

0 
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1 

-3 

2 

2 

1 


0 

1 

2 

0 

-1 


0 

0 

1 

0 

-5/12 

12R 3 +R 4 

0 

0 

0 

0 

0 


0 

0 

0 

0 

0 


1 

-3 

0 

2 

11/6 


0 

1 

0 

0 

-1/6 

(-2 )R,+R, 

0 

0 

1 

0 

-5/12 

(-2 ) R , + R, 

0 

0 

0 

0 

0 


0 

0 

0 

0 

0 


1 

0 

0 

2 

4/3 


0 

1 

0 

0 

-1/6 


0 

0 

1 

0 

-5/12 

( 3)R 2 + R , 

0 

0 

0 

0 

0 


0 

0 

0 

0 

0 



( 1 ) 


Since there are three nonzero rows (or equivalently, three leading l’s) the row space and 
column space are both three dimensional so rank (A) = 3. 

To find the nullity of/1, we find the dimension of the solution space of the linear system 
Ax = 0. The system can be solved by reducing the augmented matrix to reduced row 
echelon form. The resulting matrix will be identical to (1), except with an additional last 
column of zeros, and the corresponding system of equations will be 


x 1 + Ox 2 + 0x 3 + 2x 4 
Oxj + x 2 + 0x 3 + Ox 4 
Oxj + 0x 2 + x 3 + Ox 4 


— x 5 = 0 
3 5 


--x 5 = 0 
6 


~—x 3 =0 
12 5 


The system has infinitely many solutions: 


X] = -2 X4+(-4/3) X 5 


X 2 = (1/6) x 5 


x 3 = (5/12) x 5 


X 4 = S 


X 5 = t 

The solution can be written in the vector form: 


c 4 = (-2, 0, 0, 1 , 0) c 5 = (-4/3, 1/6, 5/12,0,1) 
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Therefore the null space has a basis formed by the set 
{(-2, 0, 0, 1, 0), (-4/3, 1/6, 5/12,0,1)} 

The nullity of the matrix is 2. Now Rank (A) + nullity (A) = 3 + 2 =5 = n 
Theorem 4 If A is an in x n, matrix, then 

(a) rank (A) = the number of leading variables in the solution of Ax = 0 

(b) nullity (A) = the number of parameters in the general solution of Ax = 0 

Example 11 Find the number of parameters in the solution set of Ax = 0 U' A is a 5x7 
matrix of rank 3. 

Solution nullity (A) = n — rank (A) = 7-3 =4 
Thus, there are four parameters. 

Example Find the number of parameters in the solution set of Ax = 0 if ,4 is a 4x4 
matrix of rank 0. 

Solution nullity (A) = n - rank (A) = 4-0 =4 
Thus, there are four parameters. 

Theorem 5 \i'A is any matrix, then rank (A) = rank (A r ) 

Four fundamental matrix spaces 

If we consider a matrix A and its transpose A r together, then there are six 
vectors spaces of interest: 

Row space of A row space of A T 
Column space of A column space of A T 

T 

Null space of A null space of A 

However, transposing a matrix converts row vectors into column vectors and column 
vectors into row vectors, so that, except for a difference in notation, the row space of/1 7 
is the same as the column space of A and the column space of A T is the same as row 
space of of A. 

This leaves four vector spaces of interest: 

Row space of A column space of A 
Null space of A null space of A T 

These are known as the fundamental matrix spaces associated with A, if A is an m x n 
matrix, then the row space of A and null space of A are subspaces of if" and the column 
space of A and the null space of/l 7 are subspaces of if"'. 

Suppose now that A is an m x n matrix of rank r, it follows from theorem (5) that A T is an 
n x m matrix of rank r . Applying theorem (3) on A and A T yields 
Nullity (A)=n-r, nullity (A T )=m-r 

From which we deduce the following table relating the dimensions of the four 
fundamental spaces of an m x n matrix A of rank r. 
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Fundamental space 

Row space of A 
Column space of A 
Null space of A 
Null space of A T 


Dimension 

r 

r 

n-r 

m-r 


Example 12 If A is a 7 x 4 matrix, then the rank of A is at most 4 and, consequently, 
the seven row vectors must be linearly dependent. If A is a 4 x 7 matrix, then again the 
ra nk of A is at most 4 and, consequently, the seven column vectors must be linearly 
dependent. 


Rank and the Invertible Matrix Theorem The various vector space concepts 
associated with a matrix provide several more statements for the Invertible Matrix 
Theorem. We list only the new statements here, but we reference them so they follow the 
statements in the original Invertible Matrix Theorem in lecture 13. 


Theorem 6 The Invertible Matrix Theorem (Continued) 

Let A be an n x n matrix. Then the following statements are each equivalent to the 
statement that .4 is an invertible matrix. 

m. The columns of A form a basis of R". 

n. Col A=R n . 

o. dim Col A = n 

p. rank .4 = n 

q. Nul A = {0} 

r. dim Nul A = 0 


Proof Statement (m) is logically equivalent to statements (e) and (h) regarding linear 
independence and spanning. The other statements above are linked into the theorem by 
the following chain of almost trivial implications: 

(g) => O) => (o) => (p) => (r) => (q) => (, d) 

Only the implication (p) => (r) bears comment. It follows from the Rank Theorem 
because A is n x n. Statements (d) and (g) are already known to be equivalent, so the 
chain is a circle of implications. 

We have refrained from adding to the Invertible Matrix Theorem obvious statements 
about the row space ofyl, because the row space is the column space of^4 r . Recall from 
(1) of the Invertible Matrix Theorem that A is invertible if and only if A r is invertible. 
Hence every statement in the Invertible Matrix Theorem can also be stated for/l 7 . 

Numerical Note 

Many algorithms discussed in these lectures are useful for understanding 
concepts and making simple computations by hand. However, the algorithms are often 
unsuitable for large-scale problems in real life. 

Rank detennination is a good example. It would seem easy to reduce a matrix to echelon 
form and count the pivots. But unless exact arithmetic is perfonned on a matrix whose 
entries are specified exactly, row operations can change the apparent rank of a matrix. 
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For instance, if the value of x in the matrix 


7 

x 


is not stored exactly as 7 in a 


computer, then the rank may be 1 or 2, depending on whether the computer treats x - 7 as 
zero. 

In practical applications, the effective rank of a matrix A is often detennined from the 
singular value decomposition of/l. 


Example 13 The matrices below are row equivalent 



~2 

-1 

1 

-6 

8 ' 


'1 

-2 

-4 

3 

-2 


1 

-2 

-4 

3 

-2 


0 

3 

9 

-12 

12 

A = 

-7 

8 

10 

3 

-10 

, B = 

0 

0 

0 

0 

0 


4 

-5 

-7 

0 

4 


0 

0 

0 

0 

0 


1. Find rank A and dim Nul A. 

2. Find bases for Col A and Row A. 

3. What is the next step to perform if one wants to find a basis for Nul A? 

4. How many pivot columns are in a row echelon form of A T ? 

Solution 

1. A has two pivot columns, so rank A = 2. Since A has 5 columns altogether, dim 
Nul A = 5 - 2 = 3. 

2. The pivot columns of ,4 are the first two columns. So a basis for Col .4 is 


{ Uj,U 2 } 


2 

1 

-7 

4 


-1 

-2 

8 

-5 


The nonzero rows of B form a basis for Row A, namely {(1, -2, -4, 3, -2), (0, 3, 
9, -12, 12)}. In this particular example, it happens that any two rows of A form a 
basis for the row space, because the row space is two-dimensional and none of the 
rows of A is a multiple of another row. In general, the nonzero rows of an echelon 
form of A should be used as a basis for Row A, not the rows of A itself. 

3. For Nul A, the next step is to perform row operations on B to obtain the reduced 
echelon form of ,4. 

4. Rank A T = ra nk A, by the Rank Theorem, because Col A T = Row A. So A T has 
two pivot positions. 


Exercises 


In exercises 1 to 4, assume that the matrix ,4 is row equivalent to B. Without calculations, 
list rank A and dim Nul A. Then find bases for Col A, Row A, and Nul A. 


"1-49 

-7l 


"1 

0 

-1 

5 ' 

-1 2 -4 

1 

,B = 

0 

-2 

5 

-6 

5 -6 10 

7 


0 

0 

0 

0 
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2. A = 


3. A = 


1 

-3 

4 

-1 

9 ' 


'1 

-3 

0 

5 

-1 

-2 

6 

-6 

-1 

-10 

,B = 

0 

0 

2 

-3 

8 

-3 

9 

-6 

-6 

-3 

0 

0 

0 

0 

5 

3 

-9 

4 

9 

0 


0 

0 

0 

0 

0 


2 

-3 

6 

2 

5 “ 


~2 

-3 

6 

2 

5“ 

-2 

3 

-3 

-3 

-4 

,B = 

0 

0 

3 

-1 

1 

4 

-6 

9 

5 

9 

0 

0 

0 

1 

3 

-2 

3 

3 

-4 

1 


0 

0 

0 

0 

0 


"1 

1 

-3 

7 

9 

-9 ' 


'1 

1 

-3 

7 

9 

-9 

1 

2 

-4 

10 

13 

-12 


0 

1 

-1 

3 

4 

-3 

1 

-1 

-1 

1 

1 

-3 

,B = 

0 

0 

0 

1 

-1 

-2 

1 

-3 

1 

-5 

-7 

3 


0 

0 

0 

0 

0 

0 

1 

-2 

0 

0 

-5 

-4 


0 

0 

0 

0 

0 

0 


5. If a 3 x 8 matrix A has rank 3, find dim Nul A, dim Row A, and rank A 1 . 

6. If a 6 x 3 matrix A has rank 3, find dim Nul A, dim Row A, and rank A 1 . 

7. Suppose that a 4 x 7 matrix A has four pivot columns. Is Col A = R 4 2 Is Nul A = R 3 7 
Explain your answers. 

8. Suppose that a 5 x 6 matrix .4 has four pivot columns. What is dim Nul A? Is Col A = 
R 4 2 Why or why not? 

9. If the null space of a 5 x 6 matrix .4 is 4-dimensional, what is the dimension of the 
column space of A? 

10. If the null space of a 7 x 6 matrix ,4 is 5-dimensional, what is the dimension of the 
column space of A? 

11. If the null space of an 8 x 5 matrix A is 2-dimensional, what is the dimension of the 
row space of A2 

12. If the null space of a 5 x 6 matrix A is 4-dimensional, what is the dimension of the 
row space of A? 

13.1f^isa7x5 matrix, what is the largest possible rank of A? If A is a 5 x 7 matrix, 
what is the largest possible rank ofi4? Explain your answers. 
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14. If A is a 4 x 3 matrix, what is the largest possible dimension of the row space of ,4? If 
A is a 3 x 4 matrix, what is the largest possible dimension of the row space of A1 Explain. 

15.1f^4isa6x8 matrix, what is the smallest possible dimension ofNul.4? 

16. If A is a 6 x 4 matrix, what is the smallest possible dimension of Nul A? 
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Lecture 26 
Change of Basis 

When a basis B is chosen for an ^-dimensional vector space V, the associated coordinate 
mapping onto R" provides a coordinate system for V. Each x in V is identified uniquely 
by its ^-coordinate vector [xj B - 

In some applications, a problem is described initially using a basis B , but the problem’s 
solution is aided by changing B to a new basis C. Each vector is assigned a new C- 
coordinate vector. In this section, we study how [x] c and [x]g are related for each x in V. 
To visualize the problem, consider the two coordinate systems in Fig. 1. 

In Fig. 1(a), x = 3bj + b 2 , while in Fig. 1 (b), the same x is shown as x = 6cj + 4c2. That 
is 


[-*-]« 


3 

1 


and 


[•*-]c 


6 

4 


Example 1 shows how to find the connection between the two coordinate vectors, 
provided we know how b / and b? are formed from ci and C 2 ■ 




Figure 1 - Two coordinate systems for the same vector space 


Example 1 Consider two bases B = {bi, b 2 } and C = {ci, C 2 } for a vector space V, 
such that 


bi=4cj+C2 and b 2 = -6c 1 + C 2 


Suppose that x = 3bi + b 2 


suppose that [.v] /; = 


3 

1 


Find Me- 


( 1 ) 

( 2 ) 


Solution Since the coordinate mapping is a linear transformation, we apply (2) 
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Me "f" ] C 

= 3[6 7 ] c +[6 2 ] c 

as a matrix equation. 


[^]c=[[*,]c [*Jc] 


From (1), 


Mc = 

from (3) [.x:] c = 


and[6 2 ] c = 


-6 

1 


4 

- 6 ' 

“ 3 " 


6 

1 

1 

1 


4 


(3) 


The C - coordinates of x match those of the x in Fig. 1. 

The argument used to derive fonnula (3) is easily generalized to yield the following 
result. 

Theorem Let B = {bi, ... , b„} and C = {ci, ..., c„} be bases of a vector space V. Then 
there exists an n x n matrix P such that 

C <—B 

Me = f P l x ] B ( 4 ) 

The columns of P are the C -coordinate vectors of the vectors in the basis B. So 

C< — B 

p b =[[*,ufg c - [*„] c ] 

The matrix P in above theorem is called the change-of-coordinates matrix from B 

C <—B 

to C. Multiplication by P converts ^-coordinates into C-coordinates. Figure 2 

C <—B 

illustrates the change-of-coordinates equation (4). 

The columns of P are linearly independent because they are the coordinate vectors of 

C <—B 

the linearly independent set B. Thus P is invertible. Left-multiplying both sides of (4) 

C <—B 

by y c P j , we obtain 

(cT*) Mr = [-^]« 
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Figure 2 Two coordinate system for V. 

Thus ^ P ^ is the matrix that converts C-coordinates into ^-coordinates. That is, 

( P ) 1 = P ( 5 ) 

Change of Basis in if - If B = {bi, ..., b,,} and E is the standard basis {ei, ... , e n j in 
R", then [bAs =bi, and likewise for the other vectors in B. In this case, P is the same 

E^B 

as the change-of-coordinates matrix Pb introduced in Lecture 23, namely, 

p .=[*, />, - K\ 

To change coordinates between two nonstandard bases in if", we need above Theorem. 
The theorem shows that to solve the change-of-basis problem, we need the coordinate 
vector of the old basis relative to the new basis. 

Example 2 Let D = {dj, d 2 , dij and F = If), f 2 , fs} be basis for vector space V, and 
suppose that// = 2dj -d 2 + d 3 ,f 2 = 3d 2 + d 3 ,f 3 = -3di + 2d$. 

(a) Find the change-of-coordinates matrix from F to D. 

(b) Find [x]o for x =// - 2 f 2 + 2f 3 . 

Solution (a) /=[[/,]» [/Jo [/Jo] 

~2~| roi r-3 - 

But [/;]«= -i , u 2 \d = 3 »[/j]i)= ° 

ij [lj |_ 2 

"2 0 -3" 

P = -1 3 0 

D<-F 

1 1 2 


(b) [x]/, = \f, - 2/, + 2f 3 ] D = | fi\ D - 2[f 2 \ D + 2 [f 3 ] D 
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H] d [f 2 ] D [f 3 ] D ] 


1 

-2 

2 


2 0-3 

-13 0 

1 1 


1 

-2 


-4 

-7 


2 

ILJ 


J 




-9" 

? b ■, = 

'-5l 

? c i = 

T 

, c, = 

"3" 

1 


-1 

' 1 

-4 

" z 

-5 


Example 3 Let b l = \ _ |, 6, = | |, c, = | . |, c, =| _| and consider the bases for 

« given by Z? = {bi, b 2 } and C = {Cj, C 2 }■ Find the change of coordinates matrix from B 
to C. 

Solution The matrix P involves the C - coordinate vectors of bi and 62- 


Let [bj] c = 


x, 


X, 


[c, C 2 ] 


C<-B 



and 

[b 2 ] c = 

Ti 



y 2 _ 

X, 



1 

= b, 

and 


. Then, by definition, 


x, 


y i 
y 2 . 


= 6 , 


To solve both systems simultaneously augment the coefficient matrix with bi and 
62 and row reduce: 


\Cl 


b, b 2 ] = 


VbAc = 

therefore 


6 

-5 


1 3 

-4 -5 
1 3 

0 I 7 
1 3 

0 1 
1 q 
0 1 


-9 -5 
1 -1 
-9 -5" 

-35 -21 


T? 7 + 47?j 


and [b 2 \ = 


-9 -5 
-5 -3 
6 4 

-5 -3 
4 
-3 




R x —3 R 2 


c F b =Wc ib 2 lc] = 


6 4 

-5 -3 


( 6 ) 
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Example 4 



“7“ 


"2l 


"4l 


“ 5 “ 

Let b, = 


, b 7 = 


, c, = 


, C-, = 


1 

-2 


-1 

~ 1 

1 

" z 

2 


and consider the bases for 


R' given by B = {bi, b 2 } and C = {ci, C 2 }. 

(a) Find the change - of- coordinates matrix from C to B. 

(b) Find the change - of- coordinates matrix from B to C. 

Solution 

and compute 


(a) Notice that P is needed rather than 
/><— c 


so 


[b, b 2 


c 2 ] = 


7 

-2 


C<—B 

2 4 5' 


1 — — — 


-1 

1 2 

2 4 

5“ 

7 7 

7 

-1 1 

2 

2 

4 

7 

7 

3 15 2 

7 

7 

2 4 

5“ 

7 7 

7 

1 -5 

- 8 _ 

0 2 

3" 

1 -5 

-8 


-R, 


0 -- — — 


2R l + R 2 


1 - - - 


- 7 -r 2 

3 2 


— R 7 + R, 
7 


P = 

B<—C 


2 3" 

-5 -8_ 

(b) By part (a) and property (5) above (with B and C interchanged), 


C<—B 


B<—C 


_ 1 

1 

1 

OO 

l 

_1 


1 

cn 

00 

1 _ 

~\ 

5 2 


-5 -2 


Example 5 



and consider the bases for 


R 2 given by B = {bi, b 2 } and C = {ci, C 2 }. 

(a) Find the change - of- coordinates matrix from C to B. 

(b) Find the change - of- coordinates matrix from B to C. 

Solution 

and compute 


(a) Notice that P is needed rather than P 

B<—C C<—B 


[b, b 2 


c 2 ] = 


1 


-2 

4 


-7 

9 


-5 

7 


3 

4 
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SO 


p 

B <—C 


5 3 

6 4 


(b) By part (a) and property (5) above (with B and C interchanged), 


P 

C<-5 


=( P ) 1 

B<-C 


"4 

-3' 


'2 

-3/2" 

-6 

5J 


-3 

5/2 _ 


Example 6 

1. Let F = {fi, f 2 } and G = {gi, g 2 } be bases for a vector space V, and let P be a matrix 
whose columns are [//]g- Which of the following equations is satisfied by P for all v 
in F? 

(i) [v]f = P[v\g (ii) [v]c = PMf 

2. Let B and C be as in Example 1. Use the results of that example to find the change- 
of-coordinates matrix from C to B. 

Solution 

1. Since the columns of P are G-coordinate vectors, a vector of the form Px must be a 
G-coordinate vector. Thus P satisfies equation (ii). 

2. The coordinate vectors found in Example 1 show that 


Hence 


P 

C<—B 


P 

B<-C 


|>T EL]- 


r=_L 

"i 

6 " 


"0.1 

0 .6" 

/ 10 

-1 

4 


-0.! 

0.4 


Exercises 


1. Let B = {bj, b 2 } and C= {ci, C 2 } be bases for a vector space V, and suppose that 
bi = 6 ci - 2 C 2 and 62 = 9 Ci -4 C 2 . 

(a) Find the change-of-coordinates matrix from B to C. 

(b) Find [x ]c for x = -3 bj + 2 62 . Use part (a). 

2. Let B = {bi, b 2 } and C = {ci, C 2 } be bases for a vector space V, and suppose that 
bi = -ci + 4 C 2 and 62 = 5 ci - 3 C 2 . 

(a) Find the change-of-coordinates matrix from B to C. 

(b) Find [x] c for x = 5 bi + 3 62 - 

3. Fet U= {uj, U 2 } and W= {wi, n> 2 } be bases for V, and let P be a matrix whose 
columns are [u 1 ] w and [ 1 / 2 ] w ■ Which of the following equations is satisfied by P for all x 
in V? 

(i) [x] u = P[x] w (ii) M w = P[x] u 

4. Let ,4 = {ai, a 2 , as )and D = {di, d 2 , dj} be bases for V, and let 

P = H L \d 2 \ A [d 3 L] . Which of the following equations is satisfied by P for all x in 

V? 

(i) [x] A = P[x]d (ii) [x] D = P[x\ A 
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5. Let A = {ai, ai, a 3 } and B = {bi, b 2 , b 3 } be bases for a vector space V, and suppose 
that a! = 4bi~ b 2 , a 2 = -bi + b 2 + bs, and as = b 2~2 b 3 . 

(a) Find the change-of-coordinates matrix from A to B. 

(b) Find [.v] B for x = 3 ai + 4 «2 + «i. 

In exercises 6 to 9, let B = {b 3 , b 2 } and C= {ci, C 2 } be bases for R 2 . In each exercise, 
find the change-of-coordinates matrix from B to C, and change-of-coordinates matrix 
from C to B. 


" 7 " 


-3" 


'l 


~-2 

_5_ ’ 

b 2 = 

-1 

■ >c i 

_-5_ 

i C 2 

2 

-f 


' 1 


V 


T 

8 _ 

,b 2 = 

_-5_ 

» c 7 = 

4 

’ C 2 = 

1 

~-6 


"2" 


"2 ' 


" 6 " 

-1 

,b 2 = 

0 

’ c i = 

-1 

,c 2 — 

_-2_ 


9. b 


1 


7 


2 


4 

-2 

,b 2 

-1 

> c i = 

1 


5 

2 


10. In P 2 , find the change-of-coordinate matrix from the basis B = {l-2t + t 2 , 3-5 t + 
4 t 2 , 2 t + 3 t 2 } to the standard basis C= {1 ,t,t 2 }. Then find the ^-coordinate vector for 
-1 + 2 t. 

11. In P 2 , find the change-of-coordinates matrix from the basis B={l — 3t,2 + t—5t, 
1 + 2 t) to the standard basis. Then write t 2 as a linear combination of the polynomials in 

B. 



1 2 -1 


-2 


-8 


-7 

12. Let P = 

-3 -5 0 

4 6 1 

’ v i = 

2 

3 

> V 2 = 

5 

2 

> v i = 

2 

6 


(a) . Find a basis [ui, U 2 , «.?} for R 3 such that P is the change-of-coordinates matrix from 
{«/, « 2 , to the basis {vi, V 2 , V 5 }. 

(b) . Find a basis {wi, W 2 , Ws) for R 3 such that P is the change-of-coordinates matrix from 
{vi, Vi, Vi} to {w u n> 2 , n> 3 }. 
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Lecture 27 

Applications to Difference Equations 

Recently Discrete or Digital data has been used preferably rather then continuous data in 
scientific and engineering problems. Difference equation is considered more reliable tool 
to analyze such type of data even if we are using a differential equation to analyze 
continuous process, a numerical solution is often produced from a related difference 
equation. 

In this lecture we will study some fundamental properties of linear difference equation 
that are considered best tool in Linear Algebra. 

Discrete-Time Signals Let S is the space of discrete-time signals. A signal in S is a 
function defined only on the integers and is visualized as a sequence of numbers, say, 

{ yk }• 

Digital signals obviously arise in electrical and control systems engineering, but discrete- 
data sequences are also generated in biology, physics, economics, demography and many 
other areas, wherever a process is measured, or sampled, at discrete time intervals. 

When a process beings at a specific time, it is sometimes convenient to write a signal as a 
sequence of the form(y 0 ,y 1 ,y 2 ,...). The tenn y k for k<0 either are assumed to be zero or 
are simply omitted. 

Discrete time signals are functions defined on integers that are sequences or it takes only 
a discrete set of values. 

For Example a Radio Station broadcast’s weather report once in a day so the sampling 
period is discrete, but sampling may be uniform or constant accordingly. 

Example 1 The crystal clear sounds from a compact disc player are produced from 
music that has been sampled at the rate of 44,100 times per second. At each 
measurement, the amplitude of the music signal is recorded as a number, say, j A . The 
original music is composed of many different sounds of varying frequencies, yet the 
sequence {j A } contains enough information to reproduce all the frequencies in the sound 
up to about 20,000 cycles per second, higher than the human ear can sense. 

Linear Independence in the Space 5* of Signals To simplify notation, we consider a 
set of only three signals in S, say, {u k },{v k } and \w k \. They are linearly independent 
precisely when the equation 

c i u k +c 2 v k +C 3 yv k = O for all k (1) 

implies that cj = C2 = cj = 0. The phrase “for all k” means for all integers positive, 
negative and zero. One could also consider signals that start with k = 0, for example, in a 
case “for all k” would mean for all integers k > 0. 

Suppose ci, c2, c_< satisfy (1). Then the equation (1) holds for any three consecutive 
values of k, say, k, k + 1 and k + 2. Thus (1) implies that 

CjU k+ i + C 2 V k+1 + c 3 w k+1 = 0 for all k 
and CjU k:+2 + C 2 V k+2 + C 3 W k+2 = 0 for all k 
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Hence ci, C 2 , C 3 satisfy 


u k 


™k 

u k + l 

V k+ 1 

™ k+ l 

U k+2 

V k+ 2 

™ k+ 2 


0 

0 

0 


for all k 


( 2 ) 


The coefficient matrix in this system is called the Casorati matrix of the signals, and the 
determinant of the matrix is called the Casoratian of and {tv*}. If for at least 

one value of k the Casorati matrix is invertible, then (2) will imply that a = C 2 = C 3 = 0, 
which will prove that the three signals are linearly independent. 


Example 2 

Solution 


Verify that l k , (-2) 1 ' and 3 k are linearly independent signals. 

- ( _ 2) * 3 * 

\ k+1 (-2) k+I 
j k+2 ^ k+2 


The Casorati matrix is 


yk+1 


\k+2 


Row operations can show fairly easily that this matrix is always invertible. However; it is 
faster to substitute a value for k - say, k = 0 - and row reduce the numerical matrix: 


1 

1 

f 


'1 

1 

ll 



1 

-2 

3 

~ 

0 

-3 

2 

r 2 - 

*1 

1 

4 

9 


1 

4 

9 



1 

1 

f 



'1 

1 

1 ' 

0 

-3 

2 

R 

5 


0 

-3 

2 

0 

3 

8 



0 

0 

10 


The Casorati matrix is invertible for k = 0. So l k , 


(-2) k and 3 k are linearly independent. 


If a Casorati matrix is not invertible, the associated signals being tested may or may not 
be linearly dependent. However, it can be shown that if the signals are all solutions of the 
same homogeneous difference equation (described below), then either the Casorati matrix 
is invertible for all k and the signals are linearly independent, or else for all k the Casorati 
matrix is not invertible and the signals are linearly dependent. 


Activity 

Verify that 2 A , (-l) k and 0 k are linearly independent signals. 

Linear Difference Equations Given scalars ao, ... , with ao and a„ nonzero, and 
given a signal {zk}, the equation 

Vit.+“1^.-7+- + a »-lJ’* + 7=*i for all k (3) 

is called a linear difference equation (or linear recurrence relation) of order n. For 
simplicity, ao is often taken equal to 1. If {zk} is the zero sequence, the equation is 
homogeneous; otherwise, the equation is non-homogeneous. 
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In simple words, an equation which expresses a value of a sequence as a function of the 
other terms in the sequence is called a difference equation. 

In particular an equation which expresses the value a n of a sequence {a n } as a function of 
the term a n _i is called a first order difference equation. 

Example 3 In digital signal processing, a difference equation such as (3) above 
describes a linear filter and ao, ... , a n are called the filter coefficients. If {y k ) is treated 
as the input and {zk} the output, then the solutions of the associated homogeneous 
equation are the signals that are filtered out and transfonned into the zero signal. Let us 
feed two different signals into the filter 

0.35y k + 2 +0.5y k + 1 +0.35y k = z k 

Here 0.35 is an abbreviation for \[2 / 4 . The first signal is created by sampling the 
continuous signal y = cos(nt / 4) at integer values of t, as in Fig. 3 (a). The discrete 
signals is {j*} = {....cos(0), cos(n / 4),cos(2n / 4),cos(3tt / 4 )...} 

For simplicity, write ±0.7 in place of ±y/~2 / 2, so that 

{ y k } = { ... ,1,0.7, 0, -0.7, -1, -0.7, 0, 0.7, 1, 0.7, 0, ....} 

k = 0 



Figure Discrete signals with different frequencies 

The following table shows a calculation of the output sequence {z k }, where 0.35(0.7) is 
an abbreviation for ( y[2 / 4).(\[2 / 2)= 0.25. The output is \y k }, shifted by one term. 


Table _ Computing the output of a filter 


K 

yu 

y k+i 

.Vk+2 

0.35y k +0.5y k +i +0.35y k+2 = Zk 

0 

l 

0.7 

0 

0.35(l)+0.5(0.7)+0.35(0)=0.7 

1 

0.7 

0 

-0.7 

0 

2 

0 

-0.7 

-1 

-0.7 

3 

-0.7 

-1 

-0.7 

-1 

4 

-1 

-0.7 

0 

-0.7 

5 

-0.7 

0 

0.7 

0 
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A different input signal is produced from the higher frequency signal y = cos(3nt 14). 
Sampling at the same rate as before produces a new input sequence: 

{w k } = {...., 1, -0.7, 0, 0.7, -1, 0.7, 0, -0.7, 1, -0.7, 0, ...} 

k = 0 

When {w k } is fed into the fdter, the output is the zero sequence. The fdter, called a low 
pass filter, lets {j*} pass through, but stops the higher frequency {w*}. 

In many applications, a sequence {z*} is specified for the right side of a difference 
equation (3) and a {y k } that satisfies (3) is called a solution of the equation. The next 
example shows how to find solution for a homogeneous equation. 

Example 4 Solutions of a homogeneous difference equation often have the form 
y k = r k for some r. Find some solutions of the equation 

yk +3 - 2y k+2 - 5y k + 1 + 6y k = 0 for all k (4) 

Solution Substitute r k forj* in the equation and factor the left side: 

r k+3 - 2r k+2 - 5r k+1 + 6r k = 0 (5) 

By synthetic division 
r k (? - 2r 2 - 5r + 6) = 0 



1 

-2 

-5 

6 

1 

0 

1 

-1 

-6 


1 

-1 

-6 

0 


So we get 

r k [(x-1)(x -x- 6)] = 0 
r k [(x-1) (x - 3x + 2x- 6)]=0 
r k [(x-1) {x(x -3)+2(x-3)}J =0 

r k (r - l)(r + 2)(r - 3) = 0 (6) 

since (5) is equivalent to (6), r satisfies the difference equation (4) if and only if r 
satisfies (6). Thus l k , (~2) k and 3 k are all solutions of (4). For instance, to verify that 3 k is 
a solution of (4), compute 

3 k+s - 2.3 k+2 - 5.3 k+1 + 6.3 k =3 k (27-18 -15 + 6) = 0 for all k 

In general, a nonzero signal r satisfies the homogeneous difference equation 
y k+n + a l y k+ „.i + + y k+ i +a„y k =0 foral \k 

if and only if r is a root of the auxiliary equation 

r n + a x r n ~ 1 + ... + a n _ x r + a n .l = 0 

We will not consider the case when r is a repeated root of the auxiliary equation. When 
the auxiliary equation has a complex root, the difference equation has solutions of the 
form s k cos kw and s k sin kw, for constants s and w. This happened in Example 3 . 
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Solution Sets of Linear Difference Equations 

Given a k , ..., a,„ consider the mapping T: S —» S that transforms a signal {j*}into a 
signal {wk} given by 

= y k + n + <h y k +n-! + ••• + a n-iy k+ l + a n y k 
It is readily checked that T is a linear transformation. This implies that the solution set of 
the homogeneous equation 

y k+ „ + a iy k+n -i +-+ a „-iy k+1 + * n y* = 0 for an k 

is the kernel of T (kernel is the set of signals that T maps into the zero signal) and hence 
the solution set is a subspace of S. Any linear combination of solutions is again a 
solution. 

Theorem If a n ^ 0 and if {z k } is given, the equation 

y k+n + a i y k+n .i + ••• + a „-iy k+ i + a „y k = z k for aii k (7) 

has a unique solution whenever y 0 , ... ,y n -i are specified. 

Proof If jo, ..., y„-i are specified, use (7) to define 

y n =z 0 -[a x y n _i+... + a n y 0 '\ 

And now that y 7 , ... ,y n are specified, use (7) to define y n + i. In general, use the 
recursion relation 

y n+k = Za - [«i-Pa + „-7 + - + a„y k ] (8) 

to define y n+k for k> 0. To define y k for k<0, use the recursion relation 

1 1 r 

y a = —z k - \y k+ „+ a x y k+n _i+... + a n _ x y k+1 ] (9) 

a n a n 

This produces a signal that satisfies (7). Conversely, any signal that satisfies (7) for all k 
certainly satisfies (8) and (9) so the solution of (7) is unique. 

Theorem The set H of all solutions of the nth-order homogeneous linear difference 
equation 

y k+n + a \y k+n -i + • ■• •+ a n~ i y k +i + a ny k = o for aiu cio> 

is an n-dimensional vector space. 

Proof We explained earlier why H is a subspace of S. For {jA-}in H, let F{j A -}be the 
vector in if" given by (yo,yi, ••• , y n -i)- It is readily verified that F:H —» if" is a linear 
transformation. Given any vector iyo,yi, ••• , y n -i) in if”, the previous theorem says that 
there is a unique signal {ja} in H such that F{y k }=(y 0 , yi, ... , y n -i )• It means that F is a 
one-to-one linear transformation of H onto if"; that is, F is an isomorphism. Thus dim H 
= dim if" = n. 


Example 5 Find a basis for the set of all solutions to the difference equation 
y k +3 ~ 2y k+2 - 5y k +i + 6y k = 0 for all k 

Solution 

Generally it is difficult to identify directly that a set of signals spans the solution space, 
but this problem is being resolved by two above key theorems, from the last theorem the 
solution space is exactly three - dimensional and the Basis Theorem describes that a 
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linearly independent set of n vectors in an n - dimensional space is automatically a basis. 
So l k , (-2) k and 3 k form a basis for the solution space. 

The standard way to describe the “general solution” of (10) is to exhibit a basis for the 
subspace of all solutions. Such a basis is usually called a fundamental set of solutions of 
(10). In practice, if we can find n linearly independent signals that satisfy (10), they will 
automatically span the //-dimensional solution space, as we saw in above example. 

Non-homogeneous Equations The general solution of the non-homogeneous 
difference equation 

y k+n + a xy k+n .i + -+ a n-xy k+ i + a „y k = z k for an k a i) 

can be written as one particular solution of (11) plus an arbitrary linear combination of a 
fundamental set of solutions of the corresponding homogeneous equation (10). This fact 
is analogous to the result in lecture 7 about how the solution sets of Ax = b and Ax = 0 
are parallel. Both results have the same explanation: The mapping x^>Ax is linear, and 
the mapping that transforms the signal {y k } into the signal \z k \ in (11) is linear. 

Example 6 Verify that the signal y k = k satisfies the difference equation 

y k +2 - 4y k+1 + 3y k = -4k for all k (12) 

Then find a description of all solutions of this equation. 

Solution Substitute k~ for j* in the left side of (12): 

(k + 2) 2 - 4(k + l) 2 + 3k 2 = (k 2 + 4k + 4) - 4(k 2 + 2k + 1) + 3k 2 = -4k 
So k 2 is indeed a solution of (12). The next step is to solve the homogeneous equation 
y k +2 - 4y k+1 + 3y k = 0 (13) 

The auxiliary equation is 

r — 4r + 3 =r 2 -3r-r+3=r(r-3)-l(r-3)= (r - l)(r— 3) = 0 

The roots are r = 1, 3. So two solutions of the homogeneous difference equation are l k 
and 3 k . They are obviously not multiples of each other, so they are linearly independent 
signals. (The Casorati test could have been used, too.) By the last Theorem, the solution 
space is two - dimensional, so 3 k and l k form a basis for the set of solutions of (13). 
Translating that set by a particular solution of the non-homogeneous equation (12), we 
obtain the general solution of (12): 

k 2 + cjl k + C23 k , or k 2 + ci + C23 k 

Reduction to Systems of First-Order Equations A modem way to study a 
homogeneous nth-order linear difference equation is to replace it by an equivalent system 
of first order difference equations, written in the form x k+! = Ax k for k = 0, 1,2, ... 

Where the vectors x k are in R" and A is an n x n matrix. 


Example 7 Write the following difference equation as a first order system: 
y k +3 - 2y k+2 - 5y k+1 + 6y k = 0 for all k 

U1 


Solution 


For each k, set x k = 


y k +1 

y k+2 
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The difference equation says that y k + 2 = 2y k+2 + 5yu+i - 6yu, so 



T*+l 


1 

o 

+ 

+ 

+ 

O 

l _ 


“0 

1 

0" 


y k 

•*-*+! 

J*+2 

= 

0 + 0 + y k+2 

= 

0 

0 

1 

= 

i 


_y k+3 _ 


_-6yk+5y k+l +2y k+ 2_ 


-6 

5 

2 


_y k+2 _ 


That is, Xk+i = Axu for all k, where A = 


0 

0 

-6 


1 

0 

5 


0 

1 

2 


In general, the equation y k + n + aiyk+n-i + ... + a„-iyk+i + a„yk = 0 for all k 
can be written as Xk+i = Axk for all k, where 




r 0 

1 

0 •• 

• 0 " 

}’k 


0 

0 

1 

0 

T/t+l 

,A = 





_yk+n-l _ 


0 

0 

0 

1 




1 

-«»-2 •' 

• - a ! 


JciZ 

Example 8 It can be shown that the signals 2 , 3 k Sin — , and 3 k Cos — are solutions 

ofy *+3 - 2y k +2 + 9y k +i - 18yk = 0. Show that these signals fonn a basis for the set of all 
solutions of the difference equation. 

Solution Examine the Casorati matrix: 

kn 


C(k) 


y Sin 


2 k 

2 k+l y + Sin 

2 k+2 3™ Sin 


k +1 rr;„ (k 1^ 


A+2 r:(k + 2f 


yeas 
y +x cos 
y +2 cos 


hi 

T 

(k 4 in 


2 

(k~Q 


Set k = 0 and row reduce the matrix to verify that it has three pivot positions and hence is 
invertible: 


*2-2/?, 



'1 0 

r 


"10 1" 

C( 0) = 

2 3 

0 

~ 

0 3-2 


4 0 

-9 


4 0-9 


"1 0 

1 " 



~ 

0 3 

-2 

*3- 

■4/?, 


0 0 

-13 
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The Casorati matrix is invertible at k = 0, so signals are linearly independent. Since there 
are three signals, and the solution space H of the difference equation has three- 
dimensions (Theorem 2), the signals form a basis for H, by the Basis Theorem. 

Exercises 

Verify that the signals in exercises 1 and 2 are solution of the accompanying difference 
equation. 

1. 2 k , (-4) k ; y k+2 + 2 y k+1 -8y k = 0 2. 3 k , (-3) k ; y k+2 -9y k = 0 

Show that the signals in exercises 3 to 6 form a basis for the solution set of 
accompanying difference equation. 

3. 2 k , (-4) k ; y k+2 + 2 y k+1 -8y k = 0 4. 3 k , (-3) k ; y k+2 -9y k = 0 

kit Inr 

5 . (-3) k , £(-3) k ; y k+2 + 6 y k+1 + 9y k = 0 6. 5 k Cos —, 5 k Sin —; y k+2 + 25 y k = 0 

In exercises 7 to 10, assume that the signals listed are solutions of the given difference 
equation. Determine if the signals form a basis for the solution space of the equation. 

kiz kit 

7. l k , 3 k Cos —, 3 k Sin —; y k+2 -y k+2 + 9y k+1 -9y k = 0 

8. (-l) k , k(- l) k , 5 k ; - 3 y k+2 - 9 y k+1 - 5 y k = 0 

9. (-l) k , 3 k ; y k +3 +y k + 2 -9y k +i - 9y k = 0 10. l k , (-l) k ; y k +4 - 2 y k+2 +y k = 0 

In exercises 11 and 12, find a basis for the solution space of the difference equation. 

n.yk+2-7 y k +i + I2y k = 0 12. 16y k+2 + 8 y k+1 - 3 y k = 0 

In exercises 13 and 14, show that the given signal is a solution of the difference equation. 
Then find the general solution of that difference equation. 

13. j* = k 2 ; y k + 2 + 3 y k+1 - 4jA=10k + 7 

14. y k = 2 - 2 k; y k+2 - (9/2) y k+1 + 2 y k = 3 k + 2 

Write the difference equations in exercises 15 and 16 as first order systems, x k +i = A x k , 
for all k. 

\5.y k+4 - 6 y k+3 + 8y k+2 + 6 y k+1 - 9y k = 0 

16.y* + 5-(3/4)y^ + (l/16)y* = 0 
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Lecture 28 


Eigenvalues and Eigenvectors 

In this lecture we will discuss linear equations of the form Ax = x and, more generally, 
equations of the form Ax = Ax , where A is a scalar. Such equations arise in a wide 
variety of important applications and will be a recurring theme in the rest of this course. 

Fixed Points 

A fixed point of an n x n matrix A is a vector x in R" such that Ax = x. Every square 
matrix A has at least one fixed point, namely x = 0. We call this the trivial fixed point of 
A. 

The general procedure for finding the fixed points of a matrix A is to rewrite the equation 
Ax = x as Ax = lx or, alternatively, as 

(/ - A)x = 0 (1) 

Since this can be viewed as a homogeneous linear system of n equations in n unknowns 
with coefficient matrix / - A, we see that the set of fixed points of an n x n matrix is a 
subspace of R" that can be obtained by solving (1). 

The following theorem will be useful for ascertaining the nontrivial fixed points of a 
matrix. 


Theorem 1 

If A is an n x n matrix, then the following statements are equivalent. 

(a) A has nontrivial fixed points. 

(b) I - A is singular. 

(c) det (I-A) = 0. 


Example 1 

In each part, determine whether the matrix has nontrivial fixed points; and, if so, graph 
the subspace of fixed points in an xv-coordinatc system. 


(a) A = 


6 

2 


0 b ) A = 


2 

1 


Solution 

(a) The matrix has only the trivial fixed point since. 


(I-A) = 


det (I-A) =det 


"i ck 


6^ 


r -2 -6^ 

vO ij 


ll 


v-1 -K 


-2 -6 

-1 -1 


= (-l)(-2)-(-l)(-6)=-4*0 


(b) The matrix has nontrivial fixed points since 
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(I-A) = 


1 0 
0 1 


det(/ - A) = det 


0 2 

7 vO b V v 
f 1 -2) 

= 0 

v o oj 



The fixed points x =(x, y) are the solutions of the linear system (I - A)x=0, which we can 


express in com] 

Donent form as 


"1 -2 

X 


" 0 " 


0 0 

y\ 


0 


A general solution of this system is 

x = 2t,y = t (2) 


which are parametric equations of the line y = \x. 
vector form of this line that the fixed points are 


X 


"2 1 

— 1 

~ 2 ~ 

y\ 


_t\ 

— 1 

1 


As a check, 


Ax = 


1- 

<N 

[ Q 

To 1 
_1 


1 

ko 

0 1 

t \ 


t 


= x 


so every vector of form (3) is a fixed point of A 


It follows from the corresponding 


( 3 ) 



Figure 1 


Eigenvalues and Eigenvectors 

In a fixed point problem one looks for nonzero vectors that satisfy the equation Ax = x. 
One might also consider whether there are nonzero vectors that satisfy such equations as 
Ax = 2x, Ax = -3x, Ax = y/2x 

or, more generally, equations of the form Ax = Ax in which 2. is a scalar. 

Definition If A is an n x n matrix, then a scalar A is called an eigenvalue of A if there 
is a nonzero vector x such that Ax =Ax . If A is an eigenvalue of A, then every nonzero 
vector x such that Ax = Ax is called an eigenvector of .4 corresponding to A . 
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Example 2 


Let A = 

Solution 


1 6 
5 2 


u = 


and v = 


3 

-2 


. Are u and v eigenvectors of^? 


Au = 


"1 6" 

6 


~-24~ 

= -4 

6 

_5 2_ 

_-5_ 


20 


-5 


= -4m 


“1 6" 

" 3 ” 


-9 

^ 2 

“3" 

5 2 

-2 


11 


-2 


Av = 


Thus u is an eigenvector corresponding to an eigenvalue - 4, but v is not an eigenvector 
of^, because Av is not a multiple of v. 


1 6 
5 2 


find the corresponding eigenvectors. 


Example 3 

Show that 7 is an eigenvalue of A = 

Solution 

The scalar 7 is an eigenvalue of A if and only if the equation 
Ax = lx 

has a nontrivial solution. But (A) is equivalent to Ax -lx = 0, or 
(A-1I) x = 0 

To solve this homogeneous equation, form the matrix 
A-1I = 


(A) 

(B) 


1 

1_ 


i 

i 

o 


1 

VO 

1_ 

I 

<N 

_i 


i 

o 

^1 

1_ 


5 -5 


The columns of A — II are obviously linearly dependent, so (B) has nontrivial solutions. 
Thus 7 is an eigenvalue of A. To find the corresponding eigenvectors, use row operations: 
-6 6 0 " 


5-5 0 
1 -1 0 
5-5 0 
1 -1 0 
0 0 0 


(* 2 - 5 *,) 


The general solution has the fonn X 2 
eigenvector corresponding to 2 = 7. 


Each vector of this fonn with x 2 ^ 0 is an 


The equivalence of equations (A) and (B) obviously holds for any 2 in place of 2=7. 
Thus 2 is an eigenvalue of A if and only if the equation 

(A - A I)x = 0 (C) 

has a nontrivial solution. 
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The set of all solutions of (A - AI)x = 0 is just the null space of the matrix A - /l /. So 
this set is a subspace of R" and is called the eigenspace of A corresponding to A . The 
eigenspace consists of the zero vector and all the eigenvectors corresponding to A . 


Example 3 shows that for matrix A in Example 2, the eigenspace corresponding to 2=7 
consists of all multiples of (1, 1), which is the line through (1, 1) and the origin. From 
Example 2, one can check that the eigenspace corresponding to A = -4 is the line through 
(6, -5). These eigenspaces are shown in Fig. 1, along with eigenvectors (1,1) and (3/2, - 
5/4) and the geometric action of the transformation x —> Ax on each eigenspace. 



FIGURE 2 Eigenspaces for A — —4 and A — 


4-16 

Example 4 Let A = 2 1 6 . 

2-18 

Find a basis for the corresponding eigenspace where eigen value of matrix is 2. 
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4 -1 6 200 2 -1 6 

Solution Form A-21 = 2 1 6-0 2 0 = 2 -1 6 and row reduce the 

2 -1 8 J |_0 0 2 j [2 -1 6 

2 -l 6 Ol augmented matrix for (A - 21) x = 0: 

2-160 
2-160 
"2 -1 6 0 " 

~ 0 0 0 0 r 2 -r, 

2-160 

"2 -1 6 0 " 

- 0 0 0 0 R 3 -R 1 
0 0 0 0 

At this point we are confident that 2 is indeed an eigenvalue of A because the equation 
(A - 27) x = 0 has free variables. The general solution is 
2xj —x 2 + 6 x 3 = 0 .(o) 

Let x 2 =t,x 2 =s then 
2xj = t- 6s 

Xj = {^y^)t-2s 

then 

Xj t/2-3s 112 -3 s 1/2 -3 

x 2 = t = t +0 =t 1 + 5 O 

x 3 s 0 s 0 1 


By back substitution the general solution is 
Xj] fl/2l [-3] 

x 2 =x 2 1 +X 3 0 ,x 2 andx 3 free 

x 3 0 1 

The eigenspace, shown in Fig. 2, is a two - dimensional subspace of R 3 . A basis is 
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FIGURE 3 A acts as a dilution Ofl <be cjgpnspaee- 


The most direct way of finding the eigenvalues of an n x n matrix A is to rewrite the 
equation Ax = Ax as Ax = AIx , or equivalently, as 

(AI - A)x = 0 (4) 

and then try to determine those values of A, if any, for which this system has nontrivial 
solutions. Since (4) have nontrivial solutions if and only if the coefficient matrix AI - A is 
singular, we see that the eigenvalues of A are the solutions of the equation 

det(/l7 -A) = 0 (5) 

Equation (5) is known as characteristic equation. Also, if A is an eigenvalue of A, then 
equation (4) has a nonzero solution space, which we call the eigenspace of A 
corresponding to A . It is the nonzero vectors in the eigenspace of A corresponding to A 
that are the eigenvectors of A corresponding to A . 

The above discussion is summarized by the following theorem. 

Theorem If A is an n x n matrix and A is a scalar, then the following statements are 
equivalent. 

(i) A is an eigenvalue of A. 

(ii) A is a solution of the equation det(fi/ - A) = 0 . 

(iii) The linear system (AI - A)x = 0 has nontrivial solutions. 

Eigenvalues of Triangular Matrices If A is an n x n triangular matrix with diagonal 
entries an, a 22 , ..., a nn , then AI - A is a triangular matrix with diagonal entries 
A-a n ,A-a 22 ,...,A-a nn . Thus, the characteristic polynomial of A is 

det (AI - A) = (A - a { l )(A - a 22 ) • • • (A - a nn ) 
which implies that the eigenvalues of A are 

A\ ~ a \V ^2 ~ a 22 ’ • • ' ’ \ ~ a nn 

Thus, we have the following theorem. 
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Theorem If A is a triangular matrix (upper triangular, lower triangular, or diagonal) 
then the eigenvalues of A are the entries on the main diagonal of A. 


Example 5 (Eigenvalues of Triangular Matrices) 


By inspection, the characteristic polynomial of the matrix A = 


\ 0 


7 - 

' Q 


-4 


0 

0 

6 

3 


is 


p(A) = (A-\)(A + f)(A-6 ) 2 . So the distinct eigenvalues of A are A = \, A = -f, and 
A = 6. 


Eigenvalues of Powers of a Matrix Once the eigenvalues and eigenvectors of a 
matrix A are found, it is a simple matter to find the eigenvalues and eigenvectors of any 
positive integer power of A. For example, if A is an eigenvalue of A and x is a 
corresponding eigenvector, then A 2 x = A(Ax) = A(Ax) = A(Ax) = A(Ax) = A 2 x , which 

shows that A is an eigenvalue of A and x is a corresponding eigenvector. In general we 
have the following result. 

Theorem If A is an eigenvalue of a matrix A and x is a corresponding eigenvector, 
and if k is any positive integer, then A k is an eigenvalue of A k and x is a corresponding 
eigenvector. 

Some problems that use this theorem are given in the exercises. 

A Unifying Theorem Since A is an eigenvalue of a square matrix A if and only if 
there is a nonzero vector x such that Ax = A x, it follows that A = 0 is an eigenvalue of A 
if and only if there is a nonzero vector x such that Ax = 0. However, this is true if and 
only if det(^) = 0, so we list the following 

Theorem If A is an n x n matrix, then the following statements are equivalent. 

(a) The reduced row echelon form of A is /„. 

(b) A is expressible as a product of elementary matrices. 

(c) A is invertible. 

(d) Ax = 0 has only the trivial solution. 

(e) Ax = b is consistent for every vector b in R n . 

(f) Ax = b has exactly one solution for every vector b in R n 

(g) The column vectors of A are linearly independent. 

(h) The row vectors of A are linearly independent. 

(i) det(/4)* 0. 

(j) A= 0 is not an eigenvalue of A. 
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(1) Is 5 an eigenvalue of A = 3 0 5 ? 

2 2 6 

(2) If x is an eigenvector for A corresponding to X , what is A 3 xl 

Solution 

(1) The number 5 is an eigenvalue of A if and only if the equation (A-XI) x = 0 has a 
nontrivial solution. Form 

"6 -3 ll [5 0 0] [1 -3 1 

A-5I = 3 0 5-050 = 3 -5 5 

2 2 6 _| |_0 0 5_| |_2 2 1 

and row reduce the augmented matrix: 

"1 -3 1 0" 

3-550 
2 2 10 

"1 -3 1 0" 

- 0 4 2 0 f ? 2 - 3/?j 

2 2 10 

"1 -3 1 0" 

- 0 4 2 0 - 2i?, 

0 8-10 
"1 -3 1 0" 

- 0 4 2 0 R 3 -2R 2 

0 0-50 

At this point it is clear that the homogeneous system has no free variables. Thus A - 51 is 
an invertible matrix, which means that 5 is not an eigenvalue ol'A. 

(2) . If jc is an eigenvector for ,4 corresponding to X , then ,4* = Xx and so 

A 2 x = A(X x) = XAx = X 2 x 

Again A 3 x = A(A 2 x) = A(X 2 x) = X 2 Ax = X 3 x. The general pattern, A k x = X k x, is 
proved by induction. 

Exercises 

1. Is 1 = 2 an eigenvalue of 
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2. Is 


-1 + V2 

1 


an eigenvector of 


2 1 
1 4 


? If so, find the eigenvalue. 


3.1s 


4. Is 


4 

-3 

1 

1 

-2 

1 


an eigenvector of 


3 7 9 

-4 -5 1 

2 4 4 


? If so, find the eigenvalue. 


an eigenvector of 


3 6 7 
3 3 7 
5 6 5 


? If so, find the eigenvalue. 


5. Is X = 4 an eigenvalue of 


6 . Is X = 3 an eigenvalue of 


3 0-1 

2 3 1 

-3 4 5 

1 2 2 ' 

3 -2 1 

0 1 1 


? If so, find one corresponding eigenvector. 


? If so, find one corresponding eigenvector. 


In exercises 7 to 12, find a basis for the eigenspace corresponding to each listed 
eigenvalue. 


1. A = 


4 -2 

-3 9 


,X =10 


8. A = 


7 4 

-3 -1 


,X =1 5 


9. A 


4 0 1 

-2 R=GL 

-2 0 1 


2,3 


10 . A 


1 0 -1 
1 -3 0 

4 -13 1 


H,= 2 - 


11 . A = 


4 2 3 

-1 IX =-3 
2 4 9 


12. A = 


3 0 2 0 
13 10 
0 110 
0 0 0 4 


A = 4 


Find the eigenvalues of the matrices in Exercises 13 and 14. 
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“0 

0 

0 " 


"4 

0 

0 " 

0 

2 

5 

14. 

0 

0 

0 

0 

0 

-1 


1 

0 

-3 


15. For A = 


1 2 
1 2 
1 2 


3 

3 

3 


find one eigenvalue, with no calculation. Justify your answer. 


16. Without calculation, find one eigenvalue and two linearly independent vectors of 



5 

5 

5 


5 

5 

5 


Justify your answer. 
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Lecture 29 

The Characteristic Equation 


The Characteristic equation contains useful information about the eigenvalues of a square 
matrix A. It is defined as 

det (A - AI) = 0, 

Where X is the eigenvalue and I is the identity matrix. We will solve the Characteristic 
equation (also called the characteristic polynomial) to work out the eigenvalues of the 
given square matrix A. 


Example 1 


Find the eigenvalues of A = 


2 

3 


3 

-6 


Solution In order to find the eigenvalues of the given matrix, we must solve the matrix 
equation 

(A - A/)x = 0 

for the scalar A such that it has a nontrivial solution (since the matrix is non singular). 
By the Invertible Matrix Theorem, this problem is equivalent to finding all A such that 
the matrix A-AI is not invertible, where 


A-AI 


"2 

3] 


"/l 

O' 


~2-A 

3 

_3 

-6 


0 

A' 


3 

-6-A 


By definition, this matrix A-AI fails to be invertible precisely when its determinant is 
zero. Thus, the eigenvalues of/1 are the solutions of the equation 


dct(/l - AI) = det 


2-/1 

3 


3 

-6-A 


= 0 . 


Recall that 
So 


det 


= ad -be 


a b 
c d 

det(A - AI) = (2 - A)(-6 - A) - (3)(3) 
= -l2 + 6A-2A + A 2 -9 
= A 2 +4A -21 


A 2 +4/1-21 = 0, 
(A-3)(A + 7) = 0, 

so the eigenvalues of ,4 are 3 and -7. 


Example 2 


Compute det A for A = 


1 

2 

0 


5 

4 

-2 


0 

-1 

0 
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Solution 

Firstly, we will reduce the given matrix in echelon fonn by applying elementary row 
operations 


by R 2 -2 R t 
"1 5 


A = 


0 -6 

0 -2 


0 

-1 

0 


by R 2 R , 

"1 5 0 

- 0-20 
0 -6 -1 
by R 3 -3 R 2 
"1 5 0 

- 0-20 
0 0-1 


which is an upper triangular matrix. Therefore, 

detA = (l)(-2)(-l) 


= 2 . 


Theorem 1 Properties of Determinants 

Let A and B be two matrices of order n then 

(a) A is invertible if and only if det A ^ 0. 

(b) det AB = (det A)(det B ). 

(c) det A t = det A. 

(d) If A is triangular, then det A is the product of the entries on the main diagonal of A. 

(e) A row replacement operation on A does not change the determinant. 

(f) A row interchange changes the sign of the determinant. 

(g) A row scaling also scales the determinant by the same scalar factor. 

Note These Properties will be helpful in using the characteristic equation to find 
eigenvalues of a matrix A. 


Example 3 


(a) Find the eigenvalues and corresponding eigenvectors of the matrix 


A = 


1 3 
4 2 


(b) Graph the eigenspaces of A in an xy-coordinate system. 


Solution (a) The eigenvalues will be worked out by solving the characteristic equation 
of A. Since 
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AI-A = A 


"1 

o' 


"1 3l 


'A- 1 

-3 " 

0 

1 


4 2 


-4 

A-2 


The characteristic equation det (AI-A) = 0 becomes 


A -1 

-4 


-3 

A-2 


= 0 . 


Expanding and simplifying the determinant, it yields 

A 2 -3A-10 = 0, 


or 

(A + 2)(A - 5) = 0. 

Thus, the eigenvalues of A are A = -2 and A = 5 . 


( 1 ) 


Now, to work out the eigenspaces corresponding to these eigenvalues, we will solve the 
system 


~A -1 

-3 " 

x 


~0~ 

-4 

A-2 

_yj 


0 


for A = -2 and A = 5 . Here are the computations for the two cases. 


(i) Case X, = -2 

In this case Eq. (2) becomes 


"-3 -3" 

x 


" 0 " 

-1 

1 

1 

_1 

_yj 


0 


which can be written as 

—3x - 3 v = 0, 

-4x-4y = 0 =>x = -y. 

In parametric form, 

x = -1, y = t . (3) 

Thus, the eigenvectors corresponding to A = -2 are the nonzero vectors of the fonn 


x = 

It can be verified as 


X 


~-t 

— / 

'-f 

_yj 


_t\ 


l 


(4) 


i 

CO 

i_ 

~-t 


1 

K) 

_i 

= -2 

~-t 

l_4 2j 

t \ 


_-2t\ 


t 


= -2x 


Thus, 

Ax = Ax 

(ii) Case X = 5 

In this case Eq. (2) becomes 


1 

CO 

i 

1_ 

X 


"0" 

l 

CO 

1 

_1 

y\ 


0 


which can be written as 
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4x - 3 v = 0 


In parametric form, 


-4x + 3y = 0 => x = — y. 


x = —t, y = t. 
4 


(5) 


Thus, the eigenvectors corresponding to A = 5 are the nonzero vectors of the form 
x 


x = 

It can be verified as 
Ax = 


y 


¥ 


= t 


( 6 ) 


1 

m 

i_ 

-1 

4^| UJ 

_1 


1 

■Hs 

_i 

= 5 

1 

_i 

i 

<N 

_ t \ 


i 

k/i 

*-+. 

i_ 


t 


= 5x. 


(b) The eigenspaces corresponding to A = -2 and A = 5 can be sketched from the 
parametric equations (3) and (5) as shown in figure 1(a). 



It can also be drawn using the vector equations (4) and (6) as shown in Figure 1(b). When 
an eigenvector x in the eigenspace for A = 5 is multiplied by A, the resulting vector has 
the same direction as x but the length is increased by a factor of 5 and when an 
eigenvector x in the eigenspace for A = -2 is multiplied by A, the resulting vector is 
oppositely directed to x and the length is increased by a factor of 2. In both cases, 
multiplying an eigenvector by A produces a vector in the same eigenspace. 
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Eigenvalues of an n x ft matrix 

Eigen values of an n x n matrix can be found in 
the similar fashion. However, for the higher values of n, it is more convenient to work 
them out using various available mathematical software. Here is an example for a 3x3 
matrix. 


Example 4 


Find the eigen values of the matrix 



-1 

0 

-17 


0 

1 

8 


Solution 


det( AI - A) = det 
/l 

= 0 
4 


"/l 

0 

0 ' 


"0 

-1 

0 " 

0 

/l 

0 

- 

0 

0 

1 

0 

0 

/l 


-4 

-17 

8 


1 0 
/l -1 

17 A-8 


= A 3 -8 A 2 + 17A-4, 

which yields the characteristic equation 

A 3 -8A 2 +17A-4 = 0 


(V) 


( 8 ) 


To solve this equation, firstly, we will look for integer solutions. This can be done by 
using the fact that if a polynomial equation has integer coefficients, then its integer 
solutions, if any, must be divisors of the constant term of the given polynomial. Thus, the 
only possible integer solutions of Eq.(8) are the divisors of-4, namely ± 1, ±2, and ±4 . 
Substituting these values successively into Eq. (8) yields that A =4 is an integer solution. 
This implies that A - 4 is a factor of Eq.(7), Thus, dividing the polynomial by A - 4 and 
rewriting Eq.(8), we get 

(A-4)(A 2 -4A + 1) = 0. 
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Now, the remaining solutions of the characteristic equation satisfy the quadratic equation 
A 2 -4A + l = 0. 


Solving the above equation by the quadratic fonnula, we get the eigenvalues of A as 
A = 4, A = 2 + y/3, A = 2-S 


Example 5 


Find the characteristic equation of A = 


5 

0 

0 

0 


-2 

3 

0 

0 


6 -1 
-8 0 
5 4 

0 1 


Solution 

get 


Clearly, the given matrix is an upper triangular matrix. Forming A - AI , we 


5-/1 


det(^ - AI) = det 


0 

0 

0 


-2 6 

3-A -8 

0 5-/1 

0 0 


-1 

0 

4 

1-A 


Now using the fact that determinant of a triangular matrix is equal to product of its 
diagonal elements, the characteristic equation becomes 

(5 - A) 2 (3 - A)(l - A) = 0. 

Expanding the product, we can also write it as 

A 4 -14A 3 +68A 2 - 130/1 + 75 = 0. 

Here, the eigenvalue 5 is said to have multiplicity 2 because (A - 5) occurs two times as a 
factor of the characteristic polynomial. In general, the (algebraic) multiplicity of an 
eigenvalue A is its multiplicity as a root of the characteristic equation. 


Note 

From the above mentioned examples, it can be easily observed that if A is an n x n matrix, 
then det (A - AI) is a polynomial of degree n called the characteristic polynomial of A. 


Example 6 The characteristic polynomial of a 6x6 matrix is A 6 - 4A 5 -12 A 4 . Find the 
eigenvalues and their multiplicities. 

Solution 

In order to find the eigenvalues, we will factorize the polynomial as 
A b -4A 5 -12A 4 

= A 4 (A 2 -4A-12) 

= A 4 (A - 6)(A + 2) 

The eigenvalues are 0 (multiplicity 4), 6 (multiplicity 1) and - 2 (multiplicity l).We 
could also list the eigenvalues in Example 6 as 0, 0, 0, 0, 6 and -2, so that the eigenvalues 
are repeated according to their multiplicities 
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Activity 

Work out the eigenvalues and eigenvectors for the following square matrix. 


A = 


5 

4 

-4 


8 16 
1 8 
-4 -11 


Similarity 

Let A and B be two n x n matrices, A is said to be similar to B if there exist 
an invertible matrix P such that 


P ~‘AP = B, 


or equivalently, 

Replacing Q by P , we have 


A =PBP~ 1 . 
Q' X BQ =A. 


So B is also similar to A. Thus, we can say that A and B are similar. 

Similarity transformation 

The act of changing A into P 1 AP is called a similarity 


transfonnation. 


The following theorem illustrates use of the characteristic polynomial and it provides the 
foundation for several iterative methods that approximate eigenvalues. 

Theorem 2 

If n x n matrices A and B are similar, then they have the same 
characteristic polynomial and hence the same eigenvalues (with the same multiplicities). 

Proof If B = P 'AP. then 

B-XI = P 'AP-XI = P a AP-XP 'P = P 1 (AP-XP) = P 1 (A -XI)P 
Using the multiplicative property (b) of Theorem 1, we compute 
det(fl - XI) = det [>* (A - XI)P~\ 

= detCR' 1 ). det(^ - XI). det (P) (A) 

Since 

det (P - 1 ). det (P) = det (P P) 

= det/ 

= 1 , 

Eq. (A) implies that 

det(R - XI) = d et(A - XI). 

Hence, both the matrices have the same characteristic polynomials and therefore, same 
eigenvalues. 

Note It must be clear that Similarity and row equivalence are two different concepts. (If 
A is row equivalent to B , then B = EA for some invertible matrix E.) Row operations on 
a matrix usually change its eigenvalues. 
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Application to Dynamical Systems 

Dynamical system is the one which evolves with the passage 
of time. Eigenvalues and eigenvectors play a vital role in the evaluation of a dynamical 
system. Let’s consider an example of a dynamical system. 


Example 7 Let A = 


.95 .03 
.05 .97 


. Analyze the long term behavior of the dynamical 


system defined by Xk+i = Axk (k = 0, 1, 2, ...), with x a = 


0.6 

0.4 


Solution The first step is to find the eigenvalues of A and a basis for each eigenspace. 
The characteristic equation for A is 
0 = det( A - AI) 

r 0.95 -/l 0.03 , 

0 = det = (.95 - A)(.91 - A) - (.03)(.05) 

0.05 0.97-T 

= 2 2 -1.92/1+ .92 
By the quadratic formula 

, 1.92±V(1.92) 2 -4(.92) 1.92±V.0064 1.92T.08 , 

2 2 

firstly, the eigenvectors will be found as given below. 

Ax = Ax, 

(Ax - Ax) = 0, 

(A-AI)x = 0. 

for A = 1 


= 1 or .92 


Y 0.95 

0.03^ 



0 "" 

(V 

1,0.05 

0.97; 



l/_ 

v x 2y 

"-0.05 

0.03 


fxY 

1 






= 0 , 

v 0.05 

-0.03 J 

V X 2 ) 

1 



which can be written as 


-0.05xj + 0.03 x 2 = 0 

0.03 3 

0.05x -0.03x, = 0 => x, = - x 1 or x, = —x, 

1 2 1 0.05 2 1 5 ■ 

In parametric form, it becomes 

x, = —t and x, = t. 

5 


for A = 0.92 
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"0.95 0.03> 


"0.92 0 v 

|V 

v 0.05 0.97 ; 


v 0 0.92, 

\ x i) 


= 0 , 


0.03 0.03 
0.05 0.05 




= 0 . 


J\ X 2J 


It can be written as 
0.03xj + 0.03x 2 = 0 

0.05X[ + 0.05x 2 = 0 
In parametric form, it becomes 
x, = t and x 2 = -t 


Thus, the eigenvectors corresponding to X = 1 and X = .92 are multiples of 


v , = 


and v 2 = 


respectively. 


The next step is to write the given Xo in terms of Vi and v 2 . This can be done because 
{vi, v 2 ) is obviously a basis for R 2 . So there exists weights c / and c 2 such that 


In fact, 

Here, 

3 1 ' 

5 -1 


Therefore, 


x 0 =c l v 1 +c 2 v 2 =[v 1 v 2 \ 


= l>i v 2 ] _1 x 0 = 


( 1 ) 


'3 r 

-1 

".60" 

_5 -V 


_- 40 _ 


1 


3 1 
5 -1 


Adj 


"3 

1 ' 

1 

r-i -n 

_5 

-lj 

8 

i 

m 

1 

_i 



_ 1 

"-1 

-f 

".60“ 


".125“ 

_ C 2 _ 

_ -8 

-5 

3 

.40 


.225 


( 2 ) 


Because v 2 and v 2 in Eq.(l) are eigenvectors of^, with Av 2 = v 2 and Av 2 = (.92) v 2 , x k 
can be computed as 


Xi =Axo = CiAvi + c 2 Av 2 (Using linearity of X -A Ax ) 

= ciVj + c 2 ( 92)v 2 (v; and v 2 are eigenvectors.) 

x 2 =Axj = cjAvj + c 2 (,92 )Av 2 = ciVi + c 2 (.92) v 2 . 


Continuing in the same way, we get the general equation as 
Xk = cjVi + c 2 (.92) k v 2 (k = 0, 1, 2, ...). 
Using ci and c 2 from Eq.(2), 


x k 



+ .225(.92/ 



(k = 0,1,2,...) 


(3) 


This explicit formula for x k gives the solution of the difference equation x k +i = Ax k . 
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As k —» oo, (.92)* tends to zero and;t* tends to 


.375 

.625 


= .125v r 


Example 8 


Find the characteristic equation and eigenvalues of A = 


1 

4 


Solution The characteristic equation is 

[1-/1 

0 = det (A - XI) = det 


-4 

2-/1 


= (l-/t)(2-/t)-(-4)(4), 


= A 2 -3A + 18, 

which is a quadratic equation whose roots are given as 


-4 

2 


a _ 3±V(-3) 2 -4(18) 

2 

_ 3±V^63 
~ 2 

Thus, we see that the characteristic equation has no real roots, so A has no real 
eigenvalues. A is acting on the real vector space R 2 and there is no non-zero vector v in 
R 2 such that A v = X v for some scalar X . 
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Exercises 

Find the characteristic polynomial and the eigenvalues of matrices in exercises 1 to 12. 


1. 


3 

1 




-3 

3 


3. 


2 1 
-1 4 



-4 

8 


5. 


5 3 
-4 4 



-2 

3 


7. 


1 0 -1 
2 3-1 
0 6 0 


8 . 


3 1 
0 2 
2 0 


9. 


4 0 0 

5 3 2 

-2 0 2 


10 . 


-1 0 
-3 4 
0 0 


1 

1 

2 



“ 6 

-2 

0" 


“5 

-2 

3 " 

11. 

-2 

9 

0 

12. 

0 

1 

0 


5 

8 

3 


6 

7 

-2 


For the matrices in exercises 13 to 15, list the eigenvalues, repeated according to their 
multiplicities. 



"4 

-7 

0 

2 “ 


"5 

0 

0 

0" 


0 

3 

-4 

6 


8 

-4 

0 

0 

13. 

0 

0 

3 

-8 

14. 

0 

7 

1 

0 


0 

0 

0 

1 


1 

-5 

2 

1 
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15. 


3 

-5 

3 

0 

-4 


0 0 0 0 
10 0 0 
8 0 0 0 
-7210 
19-23 


16. It can be shown that the algebraic multiplicity of an eigenvalue X is always greater 
than or equal to the dimension of the eigenspace corresponding to X . Find h in the matrix 
A below such that the eigenspace for X=5 is two-dimensional: 



-2 6 -1 

3 h 0 

0 5 4 

0 0 1 
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Lecture 30 


Diagonalization 

Diagonalization is a process of transforming a vector A to the form A = PDF 1 for some 
invertible matrix P and a diagonal matrix D. In this lecture, the factorization enables us to 
compute A k quickly for large values of k which is a fundamental idea in several 
applications of linear algebra. Later, the factorization will be used to analyze (and 
decouple) dynamical systems. 

The “D” in the factorization stands for diagonal. Powers of such a D are trivial to 
compute. 


Example 1 If D = 


5 0 
0 3 


,then D 2 = 



“5 

0 " 

~5 2 0 " 


D 3 = 



= 


0 

3_ 

0 3 2 



5 0 
0 3 
5 3 0 

0 3 3 


5 0 
0 3 


0 3 


and 


In general, 


D k 


5 k 0 
0 3* 


for k > 1 


The next example shows that if A = PDF 1 for some invertible P and diagonal D , then it 
is quite easy to compute A k . 

r 7 2 ] , , 

Example 2 Let A = . Find a formula for A , given that A = PDF , where 

I 11 

P = and D = 


5 0 
0 3 


Solution The standard formula for the inverse of a 2x2 matrix yields 
'2 1 


P 1 = 


-1 -1 


By associative property of matrix multiplication, 

A 2 = (PDP')(PDP') = PD (P 1 P) DP' 1 = PDIDP 1 = PDDP 1 


Again, 


Thus, in general, for k > 1, 


where I is the identity matrix. 


= PDP 1 = 

'i r 

~5 2 0 " 

"2 r 


-1 -2 

0 3 2 

-i -i 

| 

ll 

*-> 

! )A 2 ={PDF 1 )PD 2 P 1 = 


n-l 


A k = PD k P 1 = 


"i r 

"5* 

o" 

~ 2 r 

-1 -2 

0 

3 k _ 

-i -i 
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1 

m 

uo 

i_ 

“2 r 

i 

cn 

CN 

1 

IT) 

1 

_1 

-i -i ' 

2.5 k -3 k 

5*-3* 

2.3 k -2.5 k 

2.3* -5 


Activity 

Work outC 4 , given that C = PDF 1 where 



"1 O' 


'2 

0 " 

P = 

3 1_ 

, D = 

0 

1 


Remarks 

A square matrix A is said to be diagonalizable if A is similar to a diagonal matrix, that is, 
if A = PDF 1 for some invertible matrix P and some diagonal matrix D. The next theorem 
gives a characterization of diagonalizable matrices and tells how to construct a suitable 
factorization. 


Theorem 1 The Diagonalization Theorem 

An n x n matrix A is diagonalizable if and only if A has n linearly independent 
eigenvectors. 

In fact, A = PDF 1 , with D a diagonal matrix, if and only if the columns of P are n 
linearly independent eigenvectors of A. In this case, the diagonal entries of D are 
eigenvalues of A that correspond, respectively, to the eigenvectors in P. 

In other words, A is diagonalizable if and only if there are enough eigenvectors to form a 
basis of if". We call such a basis an eigenvector basis. 


Proof First, observe that if P is any nxn matrix with columns V/, ... , v„ and if D is 
any diagonal matrix with diagonal entries A l ,....,A n then 

AP = A[ Vj v 2 ... v„] = [Av 1 Av 2 ... Av„\, (1) 


while 


4 0 ••• 0 


PD = P 


0 



[\ v i X 2 v 2 



( 2 ) 


[0 0 ... ztj 

Suppose now that A is diagonalizable and A = PDF 1 . Then right-multiplying this 
relation by P, we have AP = PD. In this case, (1) and (2) imply that 

[Avj Av 2 ••• Av n ] = [\v 1 X 2 v 2 ••• \v n ] (3) 

Equating columns, we find that 

Av 1 =A 1 v 1 ,Av 2 =A 2 v 2 ,---, Av n = X n v n (4) 


Since P is invertible, its columns Vi,v„ must be linearly independent. Also, since these 
columns are nonzero, Eq.(4) shows that . ,X n are eigenvalues and v ly ..., v„ are 
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corresponding eigenvectors. This argument proves the “only if’ parts of the first and 
second statements along with the third statement, of the theorem. 

Finally, given any n eigenvectors V/, v„ use them to construct the columns of P and 
use corresponding eigenvalues A l , . ,A n to construct D. By Eqs. (1) - (3). AP =PD. This 

is true without any condition on the eigenvectors. If, in fact, the eigenvectors are linearly 
independent, then P is invertible (by the Invertible Matrix Theorem), and AP = PD 
implies that A = PDP' 1 . 


Diagonalizing Matrices 


Example 3 


Diagonalize the following matrix, if possible 



3 

-5 

3 


3 

-3 

1 


Solution To diagonalize the given matrix, we need to find an invertible matrix P and a 
diagonal matrix D such that A = PDP' 1 which can be done in following four steps. 


Step 1 Find the eigenvalues of ,4. 

The characteristic equation becomes 

0 = det(^-yL7) = -/l 3 -3/l 2 +4 
=-(A - 1)(A + 2) 2 

The eigenvalues are A = 1 and A = -2 (multiplicity 2) 

Step 2 Find three linearly independent eigenvectors of A. Since A is a 3x3 matrix and 
we have obtained three eigen values, we need three eigen vectors. This is the critical step. 
If it fails, then above Theorem says that A cannot be diagonalized. Now we will produce 
basis for these eigen values. 


Basis vector for_T = 1: 

(A-AI)x = 0 


0 

3 

1 

m 

1 

1_ 


o 

1_ 

-3 

-6 

-3 

x 2 

= 

0 

3 

3 

1 

o 

1 

& 

1 _ 


o | 


After applying few row operations on the matrix (A - AI ), we get 


0 

1 

f 

X 1 


" 0 " 

3 

3 

0 

x 2 

= 

0 

0 

0 

0 

x 3 


0 


which can be written as 

x 2 +x 3 =0 

3x, + 3 x 2 = 0 

In parametric form, it becomes 
x l = t , x 2 = -t, x 3 = t 
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1 

Thus, the basis vector for X = 1 is v l = -1 

1 


Basis vector for T = -2 

(A-XI)x = 0 

3 3 3 ^ 0 

-3 -3 -3 x 2 = 0 

3 3 3__x 3 _ 0 

which can be written as 

3xj + 3 x 2 + 3x 3 = 0 


-3xj - 3 x 2 - 3x 3 = 0 
3xj + 3 x 2 + 3x 3 = 0 


In parametric form, it becomes 




-1 

Thus, the basis for X = -2 is v 2 = 1 and v 3 

0 



We can check that {vj, V 2 , v^} is a linearly independent set. 

Step 3 Check that {vi, V 2 , v^} is a linearly independent set. 

Construct P from the vectors in step 2. The order of the vectors is not important. Using 

[1 - 1-1 

the order chosen in step 2, form P = [v } v 2 v 3 ] = -1 1 0 

1 0 1 
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Step 4 Form D from the corresponding eigen values. For this purpose, the order of the 
eigen values must match the order chosen for the columns of P. Use the eigen value X = - 
2 twice, once for each of the eigenvectors corresponding to X = -2: 


D = 


1 

0 

0 


0 

-2 

0 


0 

0 

-2 


Now, we need to check do P and D really work. To avoid computing P _/ , simply verify 
that /If = PD. This is equivalent to A = PDP ' 7 when P is invertible.We compute 



"1 

3 

3" 

"1 

-1 

-f 


"1 

2 

2' 

AP = 

-3 

-5 

-3 

-1 

1 

0 

= 

-1 

-2 

0 


3 

3 

1 

1 

0 

1 


1 

0 

-2 


'1 

-1 

-f 

"1 

0 

0" 


1 

2 

2" 

PD = 

-1 

1 

0 

0 

-2 

0 

- 

-1 

-2 

0 


1 

0 

1 

0 

0 

-2 


1 

0 

-2 


Example 4 


Diagonalize the following matrix, if possible. 



3 

-3 

1 


Solution The characteristic equation of A turns out to be exactly the same as that in 
example 3 i.e., 

0 = det(^ - XI) 

2-X 4 3 

= -4 -6-X -3 

3 3 1-X 


= -X 3 -3A 2 +4 


= -(X-l)(X + 2 f 


The eigen values are X = 1 and X = -2 (multiplicity 2). However, when we look for eigen 
vectors, we find that each eigen space is only one - dimensional. 


Basis for X = 1: 




1 

-1 

1 


Basis for X = -2: 


-1 

1 

0 
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There are no other eigen values and every eigen vector of A is a multiple of either v/ or 
v 2 - Hence it is impossible to form a basis of R 3 using eigenvectors of A. By above 
Theorem, A is not diagonalizable. 

Theorem 2 An n x n matrix with n distinct eigenvalues is diagonalizable. 

The condition in Theorem 2 is sufficient but not necessary i.e., it is not necessary for an n 
x n matrix to have n distinct eigen values in order to be diagonalizable. Example 3 serves 
as a counter example of this case where the 3x3 matrix is diagonalizable even though it 
has only two distinct eigen values. 


Example 5 


Determine if the following matrix is diagonalizable. 


A = 


5 -8 
0 0 
0 0 


1 

7 

-2 


Solution In the light of Theorem 2, the answer is quite obvious. Since the matrix is 
triangular, its eigen values are obviously 5, 0, and -2.Since A is a 3 x 3 matrix with 
three distinct eigen values, A is diagonalizable. 


Matrices Whose Eigenvalues Are Not Distinct 

If an n x n matrix A has n distinct eigen values, with corresponding eigen vectors V/,..., v 
„ and if P = [y?... v„], then P is automatically invertible because its columns are linearly 
independent, by Theorem 2 of lecture 28. When A is diagonalizable but has fewer than n 
distinct eigen values, it is still possible to build P in a way that makes P automatically 
invertible, as shown in the next theorem. 


Theorem 3 Let A be an n x n matrix whose distinct eigen values are . 

a. For 1 <k<p, the dimension of the eigen space for X k is less than or equal to the 
multiplicity of the eigen value X k 

b. The matrix A is diagonalizable if and only if the sum of the dimensions of the 
distinct eigen spaces is equal to n, and this happens if and only if the dimension of 
the eigen space for each of X k equals the multiplicity of X k . 

c. If A is diagonalizable and Bu is basis for the eigen space corresponding to \. k for 
each k, then the total collection of vectors in the sets Bj, ..., B p form an 
eigenvector basis for R" . 

Example 6 Diagonalize the following matrix, if possible. 
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5 0 

0 5 

1 4 

-1 -2 


0 

0 

-3 

0 


0 

0 

0 

-3 


Solution Since A is triangular matrix, the eigenvalues are 5 and -3, each with 
multiplicity 2. Using the method of lecture 28, we find a basis for each eigen space. 


Basis for A, = 5 : Vj = 


Basis for A = -3 : v 3 = 


-8 

4 

1 

0 

"O' 

0 

1 

0 


and v 2 


and v 4 


-16 

4 

0 

1 

0" 

0 

0 

1 


The set {vi,...,v*}is linearly independent, by Theorem 3. So the matrixP=[v/...v<d is 
invertible, and A=PDP ' 7 , where 



"-8 

-16 

0 

0" 


“5 

0 

0 

0" 


4 

4 

0 

0 

and D = 

0 

5 

0 

0 

p = 

1 

0 

1 

0 

0 

0 

-3 

0 



0 

1 

0 

1 


0 

0 

0 

-3 


Example 7 

(1) Computed where A = 


4 -3 
2 -1 


(2) Letd 


-3 12 
-2 7 


d. 


,andv 7 


Suppose you are told that Vi and 


y> 2 are eigenvectors of A. Use this infonnation to diagonalized. 

(3) Let A be a 4 x 4 matrix with eigenvalues 5, 3, and -2, and suppose that you 
know the eigenspace for A =3 is two-dimensional. Do you have enough 
information to determine if A is diagonalizable? 

Solution 

Here, det (A- A/)= A 2 -3 A +2=( A-2)( A -1). 
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The eigen values are 2 and 1, and corresponding eigenvectors are 


|, and v 2 =\ |. Next, form 

p=\: ; ,d= 

Since A = PDP A 8 = PD S P= 


2 0 
0 1 


, and P 1 = 


1 

-2 


PD S P= 

“3 

f 

-1 

K> 

00 

0 " 

' 1 -f 


2 

1 

1 0 

l 8 

-2 3_ 


3 1 

256 

0 

1 -1 

2 1 

0 

1 

-2 3 _ 

766 

-765“ 



510 

-509 




(2)Here, Av 1 = 


“-3 

12" 

“3“ 

-2 

7 _ 

1 


Av 2 = 


-3 12 
-2 7 


= l.v 15 and 


3.v, 


-1 

3 


Clearly, Vi and V 2 are eigenvectors for the eigenvalues 1 and 3 , respectively. Thus 


A = PDP _/ , where P = 


3 2 
1 1 


and D = 


1 0 
0 3 


(3) Yes A is diagonalizable. There is a basis {vi, V 2 } for the eigen space corresponding 
to A, =3. Moreover, there will be at least one eigenvector for X=5 and one for X =-2 say 
v_? and V 4 . Then {vi, V 4 } is linearly independent and A is diagonalizable , by 
Theorem 3. There can be no additional eigen vectors that are linearly independent from 
Vi to V 4 because the vectors are all in R 1 .Hence the eigenspaces for X=5 and X=-2 are 
both one-dimensional. 
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Exercise 

In exercises 1 and 2, let .4 = PDF 1 and compute A 4 . 

1 0 
0 1/2 


1. P = 


5 7 
2 3 


D = 


2 0 
0 1 


2. P = 


2 -3 

-3 5 


D = 


In exercises 3 and 4, use the factorization A = PDP' 1 to compute A k , where k represents 
an arbitrary positive integer. 


a 0 


“1 

0 " 

a 0 

“ 1 0 " 

3 (a-b) b 


_3 

1 

0 b_ 

_-3 1 


-2 12“ 


“3 

4“ 

“2 

0“ 

“-1 4 “ 

-! 5 


1 

1 

0 

1 

1 - 3 


In exercises 5 and 6, the matrix .4 is factored in the form PDF 1 . Use the Diagonalization 
Theorem to find the eigenvalues of A and a basis for each eigenspace. 



“2 

2 

f 


“1 

1 

2 “ 

"5 

0 

0 “ 

“1/4 

1/2 

1/4 “ 

5. 

1 

3 

1 

= 

1 

0 

-1 

0 

1 

0 

1/4 

1/2 

-3/4 


1 

2 

2 


1 

-1 

0 

0 

0 

1 

1/4 

-1/2 

1/4 



“4 

0 

-2" 


“-2 

0 

-1" 

"5 

0 

0“ 

‘0 

0 

1 “ 

6. 

2 

5 

4 

= 

0 

1 

2 

0 

5 

0 

2 

1 

4 


0 

0 

5 


1 

0 

0 

0 

0 

4 

-1 

0 

-2 


Diagonalize the matrices in exercises 7 to 18, if possible. 



“3 -f 


“2 

3“ 

7. 

1 5 

8. 

4 

1 



-1 4 -2 


4 2 2 

9. 

-3 4 0 

10. 

2 4 2 


-3 1 3 


2 2 4 
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"2 

2 

-f 


"4 

0 - 2 " 


11 . 

1 

3 

-1 

12 . 

2 

5 4 



-1 

-2 

2 


0 

0 5 



“ 7 

4 

16 " 


"0 

-4 - 

- 6 " 

13 . 

2 

5 

8 

14 . 

-1 

0 - 

-3 


-2 

-2 

-5 


1 

2 : 

5 



4 

0 

0 



-7 


-16 

4 


15 . 

1 

4 

0 


16 . 

6 


13 


2 


0 

0 

5 



12 


16 

1 

- 


"5 

-3 

0 91 


"4 

0 

0 

0 “ 



0 

3 

1 

-2 


0 

4 

0 

0 


17 . 

0 

0 

2 

: o 

18 . 

0 

0 

2 

0 



0 

0 

0 2 _ 


1 

0 

0 

2 
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Lecture 31 

Eigenvectors and Linear Transformations 


The goal of this lecture is to investigate the relationship between eigenvectors and linear 
transfonnations. Previously, we have learned how to find the eigenvalues and 
eigenvectors. We shall see that the matrix factorization^ = PDP ' 1 is also a type of linear 
transfonnations for some invertible matrix P and some diagonal matrix D. 

The Matrix of a Linear Transformation 


Let Kbe an ^-dimensional vector space, W an /u-dimensional vector space, and T 
any linear transformation from Kto W. To associate a matrix with T, we choose (ordered) 
bases B and C for V and W, respectively. 

Given any x in V, the coordinate vector [.v]b is in R" and the coordinate vector of its 
image, [T(jc)]c, is inf?'", as shown in Fig. 1. 


W 



Figure 1 A linear transformation from V to W 


Let B={bi, ..., b„} be the basis for V.lfx = r\bi +••• + r n b„, 

M 


then = 


and T is a linear transformation 


T(x) = T (rfij + • • • + r n b n ) = rj(b ,) + ■■■ + rj (b n ) 


( 1 ) 
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Using the basis C in W, we can rewrite (1) in tenns of C-coordinate vectors: 

[T{x)\ c = r x [T{b 1 )\ c + - + r [T(b n )] c (2) 

Since C-coordinate vectors are in if'", the vector equation (2) can be written as a matrix 
equation, namely 

[r(x)] c = M[x] b (3) 

where M = [[T(^)] C [T(b 2 )\ c ••• [T{b n )\] (4) 

The matrix M is a matrix representation of T, called the matrix for T relative to the 
bases B and C. 


T 

- >T(x) 


[x] £ 


Example 1 

Suppose that B = {bi, b 2 } is a basis for V and C= {ci, c 2 , c 3 } is a basis for W. Let 
T : V —> W be a linear transformation with the property that 

T(bj) = 3c j - 2c 2 + 5 c 3 and T(b 2 ) = 4c i + 7c 2 - c 3 
Find the matrix M for T relative to B and C. 

Solution 

Since M = [[T(bj)\ c [T(b 2 )] c ••• [T(b n )\ c ] and here 


Multiplication 


by M 

Figure 2 


-> [T(X)] C 


Hence, 



" 31 


~4~ 

[*W] C = 

-2 

and [T(b 2 )\ c = 

7 


5 


-1 
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Linear Transformations from V into V 

In general when W is the same as V and the basis C is the same as 5, the matrix 
M in (4) is called the matrix for T relative to B , or simply the 5-matrix for T, is 
denoted by [7] g . See Fig. 3. 


x 


T 


Multiplication 

[*L,--- 

b y [T\b 


Figure 3 

The 5-matrix of T : V —> V satisfies 

[J(x)] B =[T] g [.v] g , for all x in K 


>T{x) 


+ [T(x)] £ 


( 5 ) 


Example 2 

The mapping T : P 2 —> P 2 defined by T(a 0 + a x t + a 2 t 2 ) = a 1 + 2a 2 t 
is a linear transformation. 

(a) Find the 5-matrix for T, when 5 is the basis {l,t,t 2 }. 

(b) Verify that [T(/j)]b = [7]b|5]b for each p in P 2 . 

Solution 


(a) We have to find the 5-matrix of T : V —> V satisfies 
[T( x)] B = [ T] b [ , for all xinV 

Since T(a 0 +a ] t + a 2 t 2 ) = a 1 + 2a 2 t therefore 

T (1) = 0 , T (t) = 1, T (t 2 ) = 2t 
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“0“ 



T 


"0" 

ini)] B = 

0 


B 

0 

, [ T(t 2 )\ B = 

2 


0 

L 



0 

"1 ’ 


0 

1 



1 

0 

1 

1 

V 

0 



[ T] b = 

0 

0 

2 




0 

0 

0 



(b) For a general p (t) = a B +a it + 021 , we have 


See figure 4. 



a \ ] 


0 

0 

1 _ 

a 0 

[T(p)] b =[a 1 + 2a 2 t] B = 

2 a-, 

= 

0 0 2 

a \ 


0 


0 0 0 

_a 2 _ 


[ t ] b [P~\b 


T 



Figure 4 Matrix representation of a linear transformation 

Linear Transformations on if - 

In an applied problem involving if", a linear transformation T usually appears first 
as a matrix transformation, x —> Ax . If A is diagonalizable then there is a basis B for if" 
consisting of eigenvectors ol' A. Theorem below shows that in this case, the i?-matrix of T 
is diagonal. Diagonalizing A amounts to finding a diagonal matrix representation of 
x—>Ax. 


©Virtual University Of Pakistan 


409 





31-Eigenvectors and Linear Transformations 


VU 


Theorem: Diagonal Matrix Representation 

Suppose A = PDP ' , where D is diagonal n x n matrix. If B is the basis for R" 
formed from the columns of P , then D is the P-matrix of the transformation x —> Ax . 

Proof Denote the columns ofPby b\, ..., b n , so that B = {b i, b n } and 

P = [bj ... 6 n ]. In this case, P is the change-of-coordinates matrix P B discussed in 
lecture 23, where = x and [jc] b = P 'x 


If T (jc) = Ax for x in R'\ then 

m.=[[ mv, ... [7-(*jl] 

= [[Ab,\, ••• [Ab„] B \ 

= [p-'Ab, F-'Ab,,] 

= P'A[b, .. b,j 
= P' 1 AP 

Since A = PDF 1 , we have [T\b = P ~*AP = D. 


Definition of [7]g 
Since T (jc) = Ax 
Change of Coordinates 
Matrix multiplication 


( 6 ) 


Example 3 

Define T : R 2 —» R 2 by T (jc) = Ax, where A = 


. Find a basis B for R~ with the 


property that the P-matrix of T is a diagonal matrix. 


Solution 


Since the eigenvalues of matrix A are 5 and 3 and the eigenvectors corresponding to 


eigenvalue 5 is 


1 

-1 


and the eigenvectors corresponding to eigenvalue 3 is 


1 

-2 


Therefore A = PDP where P = 


1 1 
-1 -2 


and D = 


5 0 
0 3 


By the above theorem D matrix is the B matrix of T when B= 

Similarity of Matrix Representations 


-1 


1 

-2 


We know that A is similar to C if there is an invertible matrix P such that 
A = PCP therefore if A is similar to C , then C is the P-matrix of the transformation 
x —> Ax when the basis B is formed from the columns of P by the theorem 
above.(Since in the proof, the infonnation that D is a diagonal matrix was not used). 

The factorization^ = PCP is shown in Fig. 5. 
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X 

Multiplication 
by p-i 


Multiplication 
by A 


+■ Ax 

Multiplication 
by P 


Multiplication T 

Ms-► [Ax] b 

by C 

Figure 5 Similarity of two matrix representations: A = PCP 


Conversely, if T :R" —» R" is defined by T (jc) = Ax, and if B is any basis for R", then 
the 1?-matrix of T is similar to A. In fact, the calculations in (6) show that if P is the 
matrix whose columns come from the vectors in B , then [T\ B = P ~ l AP. Thus, the set of 
all matrices similar to a matrix A coincides with the set of all matrix representations of 
the transformation x —> Ax. 

An efficient way to compute a 5-matrix P ~ l AP is to compute AP and then to row reduce 
the augmented matrix [P AP] to [/ P A AP]. A separate computation of Pis 
unnecessary. 


Example 4 


Find T ( a 0 + a it + a 2 t 2 ), if T is the 


relative to B ={1, t, t 2 } is [ T] B = 


3 

0 

1 


linear transformation from P 2 to P 2 whose matrix 

4 0" 

5 -1 

-2 7 


Solution 


Let p( t) = do + ait + a 2 t 2 and compute 



3 

4 

0 

« 0 


3 a 0 + 4«j 

[T(p)\ b =[T] B [p] B = 

0 

5 

-1 

«i 

= 

5a } 


1 

-2 

7 

a 2 


a 0 —2a x +la 7 


So T (p) = (3ao+4ai)l + ( 5ai-a 2 )t + (ao-2ai + 7a 2 )t 2 . 
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Example 5 

Let A, B, C be n x n matrices. Verify that 

(a) A is similar to A 

(b) If A is similar to B and B is similar to C, then A is similar to C. 

Solution 


(a) A =(fy ] AI, so A is similar to A. 

(b) By hypothesis, there exist invertible matrices P and Q with the property that 

B = P ' 1 AP and C = Q 1 BQ. Substituting, and using facts about the inverse of a product 
we have 

C = Q'BQ = Q\P X AP)Q = {PQ)A(PQ) 

This equation has the proper form to show that A is similar to C. 
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Lecture 32 


Eigenvalues and Eigenvectors 


Definition 

A complex scalar X satisfies 
det(A- XI ) = 0 

if and only if there is a nonzero vector x in C n such that 
Ax= Xx. 

We call X a (complex) eigenvalue and x a (complex) eigenvector corresponding to X. 


, then the linear transformation x —> Ax on R 2 rotates the plane 
counterclockwise through a quarter-turn. 

Solution 


Example 1 


If A = 


The action of A is periodic since after four quarter-turns, a vector is back where it 
started. Obviously, no nonzero vector is mapped into a multiple of itself, so A has no 
eigenvectors in R 2 and hence no real eigenvalues. In fact, the characteristic equation of A 
is A 2 + 1 = 0 => X 1 = — 1 => A = +i 

The only roots are complex: A = i and A = -i. However, if we permit A to act on C 2 

then 


"0 

-f 

T 


i 

1 

0 

-i 


1 

"0 

-f 

T 


-i 

1 

0 

i 


1 




Thus i and —i are the eigenvalues, with 


1 

-i 



i 


as corresponding eigenvectors. 


Example 2 


Let A = 


0.5 

0.75 


- 0.6 

1.1 


.Find the eigenvalues of A, and find a basis for each eigen-space. 


Solution 

The characteristic equation of A is 


0 = det 


0.5-A 
0.75 


- 0.6 

l.l-A 


(0.5-T)(l. 1-/1)-(-0.6X0.75) 


= A 2 -1.6A + 1 
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' 0.5 

-0.6 


i 

o 

oo 

1 

o 

bs 

0 


-0.3 + 0.6/ 

-0.6 

0.75 

1.1 


0 

o 

oo 

1 

o 

bs 

i _’ 


0.75 

0.3+ 0.6/ 


From the quadratic formula, A = y[1.6±^/(-1.6) 2 -4] = 0.8 ±0.6/. For the eigenvalue 
A = 0.8 - 0.6/, we study 

,4-(0.8-0.6i) = 

( 1 ) 

Since 0.8 - 0.6/ is an eigenvalue, we know that the system 
(-0.3 + 0.6/)Xj - 0.6 x 2 = 0 

0.75xj + (0.3 + 0.6/)x, = 0 
has a nontrivial solution (with x\ and X 2 possibly complex numbers). 

0.75xj =(-0.3-0.6 i)x 2 

^ Xj = (-0.4- 0.8 /)x 2 

Taking x^_ = 5 to eliminate the decimals, we have x\ = —2 — 4i. A basis for the eigenspace 

"-2-4 i 

5 


( 2 ) 


comesponding to A = 0.8 - 0.6/ is v, = 

Analogous calculations for A =0 .8 + 0.6/ produce the eigenvector v 2 = 


-2 + 4/ 
5 


Check 

Compute 


Av 2 = 


' 0.5 

-0.6" 

'-2 + 4/' 


"-4 + 2/" 

0.75 

1.1 

5 


_ 4 + 3i _ 


= (0.8 + 0.6/)v 2 


Example 3 

One way to see how multiplication by the A in Example 2 affects points is to plot an 
arbitrary initial point-say, xo = (2, 0) and then to plot successive images of this point 
under repeated multiplications by A. That is, plot 


Xj = Ax 0 = 


'md 

o 

1 

o 

"2" 


'1.0' 

.75 1.1 _ 

0_ 


L5 



' 0.5 

-0.6' 

"1.0" 


-0.4 

X 

■i 

ll 

X 

II 




= 


z 1 

0.75 

1.1 

1.5 


2.4 


x 3 = Ax 2 

Figure 1 shows x 0 , ..., x 8 as heavy dots. The smaller dots are the locations ofx 9 , ..., Jtioo- 
The sequence lies along an elliptical orbit. 


©Virtual University Of Pakistan 


414 





32-Complex Eigenvalues 


VU 


*2 



Figure 1 Iterates of a point xo under the action of a matrix with a complex eigenvalue 

Of course Figure 1 does not explain why the rotation occurs. The secret to the rotation is 
hidden in the real and imaginary parts of a complex eigenvector. 

Real and Imaginary Parts of Vectors 

The complex conjugate of a complex vector x in C n is the vector x in C n whose 
entries are the complex conjugates of the entries in x. The real and imaginary parts of a 
complex vector x are the vectors Re x and Im x formed from the real and imaginary 
parts of the entries of x. 

Example 4 



" 3-1 “ 


"3l 



If X = 

i 

= 

0 

+ i 

i 


2 + 5 i 


2 


5 


then 



"31 


~-l~ 


"31 


"-ll 


3 + / 

Rex = 

0 

, Im x = 

1 

,and x = 

0 

-i 

1 

= 

-i 


2 


5 


2 


5 


i 

K> 

1 

i_’ 


If B is an m x n matrix with possibly complex entries, then B denotes the matrix whose 
entries are the complex conjugates of the entries in B. Properties of conjugates for 
complex numbers carry over to complex matrix algebra: 

rx = r x, Bx = B x, and rB = r B 
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Eigenvalues and Eigenvectors of a Real Matrix that Acts on C n 


Let A be n x n matrix whose entries are real. Then Ax = Ax = Ax. If A is an 

eigenvalue of A with x a corresponding eigenvector in C", then Ax = Ax = Ax = Ax 

Hence A is also an eigenvalue of A, with x a corresponding eigenvector. This shows 
that when A is real, its complex eigenvalues occur in conjugate pairs. (Here and 
elsewhere, we use the term complex eigenvalue to refer to an eigenvalue A = a+bi 
withh 0.) 


Example 5 


The eigenvalues of the real matrix in example 2 are complex conjugates namely, 

0.8 - 0.6/ and 0.8 + 0.6/. The corresponding eigenvectors found in example 2 are also 


conjugates: 

Example 6 


v , = 


-2-4/ 

5 


and 


v 2 = 


-2 + 4/ 
5 


= Vi 


If C = 


where a and b are real and not both zero, then the eigenvalues of C are 


a -b 
b a 

A = a±bi. Also, if r = \A I = yja 2 +b 2 , then 



Figure 2 


air -b/r 


r 0 

cos cp 

-sin (p 

b!r air 


0 r 

sin (p 

cos cp 


where <p is the angle between the positive x-axis and the ray from (0, 0) through (a, b). 
See Fig. 2. The angle cp is called the argument of A = a + bi. 
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Thus, the transformation x —> Cx may be viewed as the composition of a rotation 
through the angle cp and a scaling by |T|. 


*2 



Figure 3 A rotation followed by a scaling 

Finally, we are ready to uncover the rotation that is hidden within a real matrix having a 
complex eigenvalue. 

Example 7 


Let A = 


0.5 -0.6 

0.75 1.1 


,2 = 0.8-0.6/, and v, = 


be the 2x2 real matrix P = [Re v ; Im] = 


and let C = P ] AP = — 

20 


-2-4/ 
5 

-2 -4 
5 0 


, as in example 2. Also, let P 


" 0 4 " 

" 0.5 

-0.6" 

'-2 

-4] 


"0.8 

-0.6' 

1 

<N 

1 

1 

_i 

0.75 

1.1 

_ 5 

0 _ 


0.6 

0.8 


1 |2 

By Example 6, C is a pure rotation because 2| = (.8)“ + (.6) = 1. 
From C = P a AP, we obtain 


A = PCP A = P 


0.8 

0.6 


- 0.6 

0.8 


P 1 


Here is the rotation “inside” A! The matrix P provides a change of variable, say, x = Pu. 
The action of A amounts to a change of variable from x to u, followed by a rotation, and 
then return to the original variable. See Figure 4. 
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x 


A 


Ax 


Change of 
variable 


r 1 


P 


Change of 
variable 


+ 

u 


C 

Rotation 


-► Cu 


Figure 4 Rotation due to a complex eigenvalue 

Theorem 

Let A be a real 2x2 matrix with complex Eigen values 2 = a — b j (b A 0) and associated 
eigenvectors v in C 2 , then 

a -b 
b a 

R = [Rev hnv] 


A = PCP~\ C = 
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Lecture 33 

Discrete Dynamical Systems 


For a dynamical system described by the difference equation 
+ | = Ax^ eigenvalues and eigenvectors provide the key to understand the long term 

behavior and evolution of the system. Here the vector Xk gives infonnation about the 
system as time (which is denoted by k) passes. Discrete dynamical system has a great 
application in many scientific fields. For example, modem state space design method of 
standard undergraduate courses in control system and steady state response of a control 
system relies heavily on dynamical system. 

In this section, we will suppose that a matrix A is diagonalizable, with n linearly 
independent eigenvectors, v,, ..., v n , and corresponding eigenvalues A ,A 2 ,...A n . 

For convenience, assume that the eigenvectors are arranged so that 

|A|>|/L 2 |>...>K| 

Since { v 15 ..., v n } is a basis for R ”, any initial vector x 0 can be written uniquely as 
x 0 =c l v 1 +c 2 v 2 + ... + c n v n 
Since the vi are eigenvectors, 

X l = Ax 0 

= c 1 Av 1 +c 2 Av 2 +... + c n Av n 
= cAv l +c 2 A 2 v 2 +... + c n A n v n 


In general, 

x k =c 1 (A l ) k v l +... + c n {A n ) k v n (k = 0,1,2...) 

We will discuss in the following examples what can happen to x k if k—»oo . 


Example 1 

Denote the owl and wood rat populations at time k by x k = 


O k 

R, 


, where k is the time in 


months, O k is the number of owls in the region studied, and R k is the number of rats 
(measured in thousands). Suppose that 


O k+l ={.5)O k +{A)R k 

R M =-p-o k +(i.i)R k 

where p is a positive parameter to be specified. The (,5)6k in the first equation says that 
with no wood rats for food only half of the owls will survive each month, while the 
(1.1)/? k in the second equation says that with no owls as predators, the rat population will 
grow by 10% per month. If rats are plentiful, then (.4)i?k will tend to make the owl 
population rise, while the negative term -p.Oy measures the deaths of rats due to 
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predation by owls. (In fact, 1000 )p is the average number of rats eaten by one owl in one 
month.) Determine the evolution of this system when the predation parameter p is 0.104. 

Solution 

The coefficient matrix A for the given equations is 
0.5 0.4 

-p 1.1 

When p = .104, 

The matrix becomes 
0.5 0.4" 

-0.104 1.1 

For eigenvalues of the coefficient matrix put 
det {A-XI) =0 

The eigenvectors corresponding to the eigenvalues Tj=1.02 and T 2 =.58are 





An initial jc 0 can be written as Jto = c\v\ + c 2 V 2 then, for k > 0 , 

x k =c 1 (1.02)*v / + c 2 (.58)*v 2 = Cj( 1.02)* +c 2 (.58)* 



As k —> oo, (.58)* rapidly approaches zero. Assume c x * 0. Then, for all sufficiently large 
k, jc k is approximately the same as ci(1.02) k vi, and we write 


x k ~ (1 -02)* 



( 1 ) 


The approximation in (1) improves as k increases, and so for large k, 

x k+1 ~ Cj(1.02) i+1 ™ = (1.02^(1.02/ «l.02x t 

This approximation says that eventually both Xk entries i.e owls and rats grow by a factor 
of almost 1.02 each month. Also equation (1) shows that Xk is approximately a multiple 
of (10,13).So we can say that the entries in Xk are almost in the same ratio as 10 to 13.It 
means that for every 10 owls there are about 13 thousand rats. 


Trajectory of a dynamical system 


In a 2x2 matrix, geometric description of a system’s evolution can enhance the algebraic 
calculations. We can view that what happens to an initial point xo in R 2 , when it is 
transfonned repeatedly by the mapping x—»Ax. The graph of xo, xi.. .is called a trajectory 
of the dynamical system. 
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Example 2 

Plot several trajectories of the dynamical system jCk+i = Axu, when 


A = 


.80 

0 


0 

.64 


Solution 


In a diagonal matrix the eigen values are the diagonal entries and as A is a diagonal 
matrix with diagonal enteries 0.8 and 0.64,Therefore the eigenvalues of A are .8 and .64, 


with eigenvectors v, = 



~l~ 


"0" 

V; = 


and v, = 


1 

o_ 

Z 

1 


If Xo = civi + C 2 V 2 , then x k = q(.8) A 


1 

0 


+ c 2 (.64) a 


0 

1 


Of course, Xk tends to 0 because (.8) k and (.64) k both approach to 0 as k — » go. But the 
way jc k goes towards 0 is interesting. Figure 1 shows the first few terms of several 
trajectories that begin at points on the boundary of the box with comers at (±3, ±3). The 
points on a trajectory are connected by a thin curve, to make the trajectory easier to see. 


X 2 



Figure 1 The origin as an attractor 

In this example, the origin is called an attractor of the dynamical system because all 
trajectories tend towards O. This occurs whenever both eigenvalues are less than 1 in 
magnitude. The direction of greatest attraction is along the line through O and the 
eigenvector V 2 as it has smaller eigenvalue. 


©Virtual University Of Pakistan 


421 





33-Discrete Dynamical Systems 


VU 


Example 3 


Plot several typical solution of the equation x k+l = Ax k , when 


A = 


1.44 

0 


0 

1.2 


Solution 

In a diagonal matrix the eigen values are the diagonal entries and as A is a diagonal 


1.2, with eigenvectors v } = 
As 



'll 


"0" 

V, = 


and v, = 


1 

0_ 

Z 

1 


x 0 =c,v l +c 2 v 2 



'll 


'O' 



+ Cry 


1 

0 

z 

1 


, then 

x, = Cj(1.44)* 


+c 2 ( 1.2)* 


Both terms grow in size as k—>oo ,but the first term grows faster because it has larger 
eigenvalue. Figure 2 shows several trajectories that begin at points quite close to 0. 


x i 



Figure 2 The origin as a repellor 

In this example, the origin is called a repellor of the dynamical system because all 
trajectories tend away from O. This occurs whenever both eigenvalues are greater than 1 
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in magnitude. The direction of the greatest repulsion is the line through 0 and that eigen 
vector which has a larger eigenvalue. 


Remark 


From the above two examples it may be noticed that when eigenvalues are less than 1 in 
magnitude, the origin behaves as an attractor and when the eigenvalues are greater than 1 
in magnitude, the origin behaves as a repellor. 


Example 4 


Plot several typical solutions of the equation 
34.-, =£>34, where 


D = 


2.0 

0 


0 

0.5 


Show that a solution j y k } is unbounded if its initial point is not on the x 0 -axis. 


Solution 

Mistakes 


The eigenvalues of D are 2.0 and 0.5, with eigenvectors 



V 


" 0 " 

V, = 


and v, = 


1 

0 _ 

Z 

1 


If 

To = C A V , +c 2 fv 2 


To = 



, then 


T, = <4(2.0/ 


1 

0 


+ c 2 (0.5)* 


0 

1 


If v 0 is on the x 2 -axis,then q =0 and y k —>0 as k—> oo .But if y 0 is not on the x 2 - 
axis, then the first tenn in the sum for y k becomes arbitrarily large, and so \y k } is 
unbounded. Figure 3 shows ten trajectories that begin near or on the axis. 
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FIGURE 3 The origin as a saddle point. 


In this example ‘O’ is called the saddle point because one eigenvalue is greater than ‘ 1 ’ 
in magnitude and one is less than ‘ 1 ’ in magnitude. The origin attracts solution from 
some directions and repels them in other directions. 

Change of Variable 

The preceding three examples involved diagonal matrices. To handle the nondiagonal 
case, we return for a moment to the n x n case in which eigenvectors of A form a basis 
{v 7 ,-,v„} for R". Let P = \ vj v n ], and let D be the diagonal matrix with the 

corresponding eigenvalues on the diagonal. Given a sequence {jc*} satisfying x k+t = Ax k , 
define a new sequence {jk} by y k = P Vk, or equivalently, x k = Py k . 
Substituting these relations into the equation jc k +i = Ax k, and using the fact that 
A = PDP we find that Py k +i = APy k = ( PDP ~')Py k = PDy k 
Left-multiplying both sides by P , we obtain j k +i = L)j k 
If we write jk as y(k) and denote the entries in y(k) byji(^), ...,y n (k), then 


y,(k+\) 


~A, 0 ••• 0 " 

y,(k) 

y 2 (k + l) 


o t 2 ; 

y 2 ( k ) 



; o 


_y„(k+D_ 


i 

o 

o 

_ 1 

_y n (k) 


The change of variable from Xk to y k has decoupled the system of difference equations. 
The evolution of y/(k), for example, is unaffected by what happens to yi(k), ..., y n (k), 
because y 1 (k + \) = \.yj(k) for each k. 
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Example 5 

Show that the origin is a saddle point for solutions of JCk+i = Atk. where 


A = 


1.25 

-.75 


-.75 

1.25 


find the directions of greatest attraction and greatest repulsion. 

Solution 

For eigenvalues of the given matrix put 
det(^ -/l/)=0 


we find that A has eigenvalues 2 and .5, with corresponding eigenvectors v, = 


1 

-1 


and 


^2 = 


, respectively. Since 2 > 1 and .5 < 1, the origin is a saddle point of the 


dynamical system. Ifjto = civi + cjVz, then 
x k =c x {2) k Vj+c 2 {.5) k v 2 


In figure 4, the direction of the greatest repulsion is the line through v x , the eigenvector 
whose eigenvalue is greater than 1. 

The direction of the greatest attraction is determined by the eigenvector v 2 whose 
eigenvalue is less than 1. 

A number of trajectories are shown in figure 4. When this graph is viewed in terms of 
eigenvector axes, the diagram looks same as figure 4. 


y 



FIGURE 4 The origin as a saddle point. 
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Complex Eigenvalues 

Since the characteristic equation of nx n matrix involves a polynomial of degree n, the 
equation always has exactly n roots, in which complex roots are possibly included. If we 
have complex eigenvalues then we get complex eigen vectors which may be divided into 
two parts: which is real part of the vector and imaginary part of the vector. 

Note 

If a matrix has two complex eigenvalues in which absolute value of one eigenvalue is 
greater than 1 ,then origin behaves as a repellor and iterates of x 0 will spiral outwards 
around the origin .If the absolute value of an eigenvalue is less than 1 ,then the origin 
behaves as an attractor and iterates of x 0 will spiral inward towards the origin. 


Example 6 

Let 

0.8 0.5 

- 0.1 1.0 


A = 


1 ± 2/ 
1 


has eigenvalue 0.9 ±0.2 i with eigenvectors 
Find the trajectories of the system x k+] = Ax k with initial vectors 

r o i 

and 


0 

2.5 


0 

-2.5 


Solution 


Given a matrix 


A = 


0.8 

- 0.1 


0.5 

1.0 


The eigenvalues of the matrix are 0.9 + 0.2/ and corresponding eigenvectors are 


also the initial vectors are given as 


"0 

"3” 

and 

" 0 

_2.5_ 

L°_ 


_-2.5_ 



1 ± 2 / 
1 
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Example 7 

Suppose the search survival rate of a young bird is 50%,and the stage matrix A is 


A = 


0 

0.3 

0 


0 

0 

0.71 


0.33 

0 

0.94 


What does the stage matrix model predict about this bird? 


Solution 


Given a matrix A= 


0 

0.3 
0 

for eigenvalues of A we 


0 0.33 

0 0 
0.71 0.94_ 
will put det(A- A I)=0 


det(^4 - AI) 


—A 0 0.33 

0.3 —A 0 
0 0.71 0.94-A 


= -A(-0.94 + A 2 ) + 0.33(0.3 x 0.71) 
= 0.94 A -A 3 +0.07029 


for eigenvalues put det( A - AI) = 0 
0.94 A -A 3 +0.07029 = 0 
A 3 - 0.94 A -0.07029 = 0 


it gives 3 values of which are approximately 1. 01,-0.03+0.26i,-0.03-0.26i 
eigen vector for first valuefofvill be v =(,10,3 ,31) and for the next two values of 

X we will get complex eigen vectors denoted by v 2 and v 3 . 


Now 

x k =c 1 (1.01)*v 1 +c 2 (-0.03 +0.26i)* v 2 + c 3 (-0.03-0.260* v 3 

As k—>oo ,the second two vectors tends to O.So x, becomes more and more like the (real) 

vector CjO.Ol/vj .Thus the long term growth rate of the owl population will be 1.01,and 

the population will grow slowly. The eigenvector describes the eventual distribution of 
the owls by life stages: For every 31 adults, there will be about 10 juveniles and three 
subadults. 
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Exercise 


Suppose that a 3x 3 matrix A has eigenvalues 2,2/5,3/5 and corresponding eigenvectors 



' 1' 


~- 2 ~ 


"-3" 

V 1 = 

0 

V 2 = 

1 

V 3 = 

3 


-3 


-5 


7 


also x 0 = 


-2 

-5 

-3 


Find a general solution of the equation x k+l = Ax k also find what happens to 
x k when k —» oo . 
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Lecture 34 

Applications to Differential Equations 

Differential equation 

A differential equation is any equation which contains derivatives, either ordinary 
derivatives or partial derivatives. Differential equations play an extremely important and 
useful role in applied mathematics, engineering, and physics. Moreover, a lot of 
mathematical and numerical machinery has been developed for the solution of 
differential equations. 


System of linear differential equations 

A system of linear differential equations can be expressed as: 

=a n x l +... + a u x n 


x„ = a „,x + ... + ax„ 

n n 11 nn n 


where Xj(t) is a function of time, i = 1,. . . n, and the matrix of constant coefficient is 

a= M 

In many applied problems, several quantities are continuously varying in time, 
and they are related by a system of differential equations: 

x'i =a n x x + ... + a ln x n 

x 2 =a 2l x 1+ ... + a 2n x n 


Here x v x 2 ,...,x n are differentiable functions of t ,with derivatives ,and thea (> 

are constants. Write the system as a matrix differential equation 
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x = Ax 
Where 



X, (0 


x(0 


a n ... a ln 

x(t) = 


,x’(0 = 


, and A = 







a , a 

_ n\ nn _ 


Superposition of solutions 

If u and v are two solutions of x = Ax, then cu + dv is also a solution. 

Proof: 

If u and v are two solutions of x = Ax then u' = Au and v' = A v .Therefore, 

(cu + dv) = cu + dv 

= cAu + dAv 
= A(cu + dv) 

Linear combination of any set of solutions is also a solution. This property is called 
superposition of solutions. 


Fundamental set of Solutions 


If A is an nx n matrix, then there are n linearly independent functions in a 
fundamental set, and each solution of x = Ax is a unique linear combination of these ‘n’ 
functions. So we can say that fundamental set of solutions is a basis for the set of all 
solutions of x = Ax, and the solution set is n-dimensional vector space of functions. 

Initial value problem 

If a vector x 0 is specified, then the initial value problem is to construct the unique 
function x such that x = Ax and x(0) = x 0 


Example 1 

For the system x = Ax .What will be the solution when A is a diagonal matrix and is 
given by 


A = 


f 3 


0 " 

- 5 , 
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Solution 

The solution of the given system can be obtained by elementary calculus. Consider 


1 

1_ 


"3 

1 

o 

1 

1_ 

1 

£- 

1 _ 


1 

O 

-1 

1 

l 

X 

to 

1 _ 


from here we have 

x 1 ( t ) = 3xj ( t ) and x 2 ( t ) = -5x 2 ( t ) 

first of all we will find solution of x l (t) = 3x l (t) by using calculus. 

x, ( t ) = 3xj ( t ) 

A = 3X] 

dt 1 

dx, „ , 

— L = 3dt 

Xi 

Integrating both sides 

f^ = 3f dt 

J Xj J 

In Xj = 3/ + In q 
In Xj - In q = 3f 

ln(—) = 3/ 

c i 

taking anti log on both sides 
C 

Xj (t) = c x e 3 ' 

Similarly Solution of 
x 2 (/) = -5 x 2 (7) 
tv/// /te 
x 2 (/) = c 2 A 5 ' 


And it can be written in the form 


xft) 


c x e 3> 


'll 

3 1 

'o' 


= 


= C, 


e + c 9 


_x 2 (t) \ 


-51 

i C 2 e } 

i 

0_ 

z 

1 


The given system is said to be decoupled because each derivative of the function depends 
only on the function itself, not on some combination or coupling of x x (t ) and x 2 (t). 
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Observation 


From the above example we observe that a solution of the general equation x = Ax might 
be a linear combination of functions of the form 

x(t) = ve Al 

Where A is some scalar and v is some fixed non-zero vector. 

x(t) = ve At .(1) 

differentiating w.r.t 7' 
x ( t ) = Ave A ‘ 

Multiplying both sides of (1) by A 
Ax(t ) = Ave Al 

Since e /J is never zero,x (t) will equal to Ax(t) iff Av = Av. 

Eigenfunctions 

Each pair of eigenvalue and its corresponding eigenvector provides a solution of the 
equation x = Ax which is called eigenfunctions of the differential equation. 

Example 2 

The circuit in figure 1 can be described by 


"v;(o‘ 


~~(R l +R 2 )/C l 

r 2 /c ] " 

vft) 

v' 2 (t)_ 


r 2 /c 2 

-r 2 /c 2 

vft) 


Wherey (t) and v 2 (/) are the voltages across the two capacitors at time t. 

F? 1 



Figure 1 

Suppose that resistor R\ is 2 ohms, R 2 is 1 ohms, capacitor C\ is 2 farads, and capacitor 
C 2 is 1 farad and suppose that there is an initial charge of 5 volts on capacitor C 1 and 4 
volts on capacitor C 2 . Find formulas for vft) and v 2 (t) that describe how the voltages 
change over time. 

Solution 


For the data given, we can form A as 
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A= ~(R l +R 2 )/C l R 2 / Cj 

rjc 2 -r 2 /c 2 

R 1 = 2 

R 2 = 1 

Cl =2 
C 2 =1 so 

-{R l +R 1 )ic x =~^ 2+i y^^-y 2 ^-\.5 

R 2 /C x =Y 2 ^ 0.5 


-r 2 /c 2 =~Y =>-l 



-1.5 

.51 


V, 


“5“ 

A = 



,x = 

1 

, and x n = 



1 

-1 


_V 2 J 

7 U 

4 


. The vector jt# lists the initial values of jc. From 
^4, we obtain eigenvalues f = -.5 and X 2 = -2, by using det (A-kI)=0 with corresponding 
eigenvectors v, = 

_ Z J 

The eigenfunctions xft) = v / e il ' and x 2 (t) = v 2 e 2 -‘ both satisfy x' = Ax, and so does any 
linear combination of Xi and * 2 . Set 
x(t) = + c 2 v,e' ?2f 


'll 

and v, = 


2 

I 

1 


= c, 


6 + C, 


-1 

1 


■( 1 ) 


and note that jc( 0) = civi + C 2 V 2 . Since v\ and V 2 are obviously linearly independent and 
hence span R 2 , c\ and c 2 can be found to make jc( 0) equal to *o. 

We can find c) and c 2 as 


= [fi 


v 2 x 0 . 


1 -1 5 

2 1 4 

1 0 3 

0 1 -2 


hence c, = 3 and c 2 = -2 


row reduction of this matrix gives 


Put the value of q and c 2 in(l) 


x(t) = 3 


-. 5 1 r\ 

e -2 


-1 

1 


-a 


Thus the desired solution of the differential equation x' = Ax is 
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40 = 


"3A°' 5, 1 


-2 A 2 '” 

1 

OS 

1 

p 

1_ 

1 

1 

<N 

1 

(N 

_ i 


or 


J L 


J 



~3e~°' 5t +2e~ 2 '~ 

1 

1 


6e~ 0 5t - 2e~ 2t 


As both eigenvalues are negative so origin in this case behaves as an attractor and the 
direction of greatest attraction will be along the more negative eigenvalue X = -2 . 



Example 3 

Suppose a particle is moving in a planar force field and its position vector x satisfies 
x = Ax and x(0) = x 0 , where 

2.9" 

2 . 6 

Solve this initial value problem for t > 0, and sketch the trajectory of the particle. 

Solution 


A = 


4 -5 

-2 1 


The eigenvalues of the given matrix can be found by using det(A-kl) 

r-s" 

out to be X l = 6 and X 2 =-1 with corresponding eigen vectors Vj = 


=0 which are turned 

T 


and v, = 


For any constants c x and c 2 ,the function 
x{t) = c 1 v l e At> + c 2 v 2 e A2> 


c i 

“-5“ 

6 1 . ^ 

6 + C 9 

T 

t 

2 

z 

l 
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We have to find c, and c 2 to satisfy x(0)=x 0 ,so 



~-5~ 


T 

c. 


+ C 9 


1 

2 

L 

1 


2.9 

2.6 


-5c, 

2c, 


+ 



-5c, + c 2 
2c, + c 2 

2.9 = -5c, +c 2 . 

2.6 = 2c, +c 2 . 

Subtracting 1 from 2 we get 


( 1 ) 

•( 2 ) 


3 188 

c, =-— and substituting this value of c, in (1) we get c 2 = 


x(t)=~ — 
70 


“-5“ 

OC 

oo 

+ 

T 

2 

70 

l 



Figure 3 The Origin as a saddle point 

As the matrix A has both positive and negative eigenvalues so in this case origin behaves 
as a saddle point. The direction of greatest repulsion is the line through v, and 0 
corresponding to the positive eigenvalue. The direction of greatest attraction is the line 
through v 2 and 0 , corresponding to the negative eigenvalue. 
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Decoupling a Dynamical System 

For any dynamical system described by x' = Ax when A is diagonalizable, the 
fundamental set of solutions can be obtained by the methods that we have discussed in 
examples 2 and 3. 

Suppose that the eigenfunctions for A are 

with Vi, v„ linearly independent eigenvectors. Let P = [v i ... v n ] and let D be the 
diagonal matrix with entries \,...,A n , so that A = PDP Now make a change of 
variable, defining a new function y by 


y(t ) = P' 2 x(t), or equivalently, x(t) = Py(t) 
Substituting these relations into x' = Ax gives 

Py\t) = {PDP- l )Py{t ) 

Py'(t) = PDy(t ) 

pre multiplying both side by P 1 
P X Py'(t) = P l PDy{t) 
y'(t) = Dv(t) or 
y' = Dy 


y[(t) 


"4 o ••• 0" 

Ti(0 

ytt) 


0 t 2 : 

y 2 (t) 



; o 


y'n(t) \ 


i 

o 

o 

_1 

y n (t). 


The change of variable from x to y has decoupled the system of differential equations 
because the derivative of each scalar function y k depends only on j k - Since y\ = \y v we 


have y 1 (t) = c l e Zl ‘, with similar formulas for y 2 , ...,j„-Thus 


= y(0) = p- 1 x(0) = p- , x 0 


To obtain the general solution* of the original system, compute 
*(0 = iM0 = K v ,,]X0 = W^' +• •■+c„v 
This is the eigenfunction expansion as constructed in example 2. 



i 

_i 



y(t) = 

At 

, where 



c e " 

n 


_ C 2_ 


Complex Eigenvalues 

As we know that for a real matrix A , complex eigenvalues and associated eigen 
vectors come in conjugate pairs i.e if Tis an eigenvalue of A thenT will be the 2 nd 
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eigenvalue. Similarly if one eigen vector isv then the other will bev .So there will be two 
solutions of x = Ax and they are 
Xj ( t ) = ve Al , x 2 (t) = ve Al 


Here x 2 ( t ) = x, ( t ) 


1 


Re(ve ) = — Xj (t) + x 1 ( t) 


1 


Im(ve ) = — x, (t) - Xj ( t) 
2 i L 


Frorn Calculus 


x x x 

6 — 1 + X 4-h ... H-h ... 


2 ! 


nl 


(At) 2 


m n 


e = 1 + At + -——— + +... 

2! n! 

If we write A = a + ib 

e {a+lb)t = e at ,e ib ' = e a ' (cos bt + / sin bt) 


So 

ve M = (Re v + i hn v).e at (cos bt + i sin bt) 

ve Al =Rev.e°' cos bt + Re v.e at i sin bt + i\mv ,e a! cos bt - Im v.e at sin bt 
ve Al = e at (Re v cos bt - Im v sin bt) + ie a ‘ (Re v sin bt + hn v cos bt) 

So two real solutions of x = Ax are 

y 1 (t) = Re Xj (0 = [(Re v) cos bt - (hn v) sin bt] e at 

y 2 (t) = Imx, (t) = [(Rc y)sin hi + (Im v) cos bt]e ar 


Example 4 

The circuit in figure 4 can be described by the equation 


i' L 1 


- R i L 

-l/L 

h 

v' c _ 


. i/c 

-1/(R,C) 

v c_ 
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O 




Figure 4 


Where //. is the current passing through the inductor L and V c is the voltage drop across 
the capacitor C. Supposed; is 5 ohms, R? is .8 ohm, Cis .1 farad, andZ is .4 henry. 

Find formulas for ii, and v c , if the initial current through the inductor is 3 amperes and the 
initial voltage across the capacitor is 3 volts. 


Solution 

From the data given, We can form A as 


_\-R 2 /L -1 /L 

A = [ 1/C -1 /(R t C) 

R 1 = 5 
R 2 = 0.8 
C = 0.1 
1 = 0.4 



A = 


-2 -2.5 
10 -2 
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det (A - AI) 


-2-A -2.5 
10 -2-A 


= (-2-A) 2 +25 
= 4 + A 2 +4A + 25 


= A 2 +4/1 + 29 


put det (A - AI ) = 0 
A 2 +4A + 29 = 0 


the eigenvalues of A is -2 + 5 i with corresponding eigenvectors v, = 
complex solutions of 

x' = Ax are complex linear combinations of 

i 


and 


-i 

2 


The 


Xj(t) = 


,(- 2+5 i)t 


-21 5 it 

e .e 


and x 2 (t) = 


-i 


,(- 2-5 i)t 


X,{t) = 


e “'(cos5t + z'sin5t) 


By taking real and imaginary parts of jci, we obtain real solutions: 


T ,(0 = 


-sin5t 

2cos5t 


e 2r ,y 2 (t) = 


cos 5 1 
2sin5t 


-it 


Since y i and >+ are linearly independent functions, they form a basis for the two- 
dimensional real vector space of solutions = Ax. Thus, the general solution is 


x(t) = q 

To satisfy x(0) = x 0 = 


-sin5t 

2cos5t 


£ + C, 


cos5t 

2sin5t 


-it 


we need sci and c 2 


"0” 

+ C 9 

'll 

= 

"3" 

2 

z 

0_ 


_3_ 







0 

2c, 


+ 


Thus 


0 + c, — 3 
2Cj + 0 = 3 



©Virtual University Of Pakistan 


439 





34-Applications to Differential Equations 


VU 


x(t) = Cj 

-sin5t 

2cos5t 

e 21 +c 2 

cos5t 

2sin5t 


-sin5t 

A 2 '+3 

cos5t 

= 1.5 

2cos5t 

2sin5t 


-1.5sin5te 21 


3cos5te 21 

3 cos 5 te~ 2 ‘ 

+ 

6sin5te 2 ' 


-1.5sin5te 21 + 3cos5te 21 


3 cos 5 te 2 ‘ +6sin5te 


4(0 


-1.5sin5t + 3cos5t 

_v c (0_ 


3cos5t + 6sin5t 



Figure 5 The origin as a spiral point 


In this figure, due to the complex eigenvalues a rotation is caused by the sine and cosine 
function and hence the origin is called a spiral point. The trajectories spiral inwards as in 
this example the real part of the eigenvalue is negative. 

Similarly, it is important to note that when A has a complex eigenvalue with positive real 
part, the trajectories spiral outwards .And if the real part of the eigenvalue is zero, then 
the trajectories form ellipses around the origin. 
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Exercise 1 


The circuit in given figure can be described by 

_( 1/ 4- f 


v[(t) 

v' 2 (t) 


/ R l + /R? /Cl 


1/ 


%c 2 


1 

- 1 . 


R 2 C x 


%C 2 


Vj(0 

v 2 (t) 


Where (/) and v 9 (t) are the voltages across the two capacitors at time t. 

F? 1 



Suppose that resistor R \ is 1 ohms, R 2 is 2 ohms, capacitor Ci is 1 farad, and capacitor C 2 
is 0.5 farad, and suppose that there is an initial charge of 7 volts on capacitor C 1 and 6 
volts on capacitor CY Find fonnulas for vi(t) and V 2 (t) that describe how the voltages 
change over time. 

Exercise 2 


Find formulas for the currentand the voltage V c for the circuit in Example 4, assuming 
that R x = 10 ohms,R 2 =0.2 ohm , C = 0.4 farad, L = 0.2 henry 
the initial current is 0 amp, and the initial voltage is 16 volts. 
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Lecture 35 

Iterative Estimates for Eigenvalues 

Eigenvalues and eigenvectors 

Let A be an n x n square matrix, X is any scalar value. If there exists a non-zero 
vector X, such that it satisfies the equation AX = XX then X is called the eigenvalue and 
X is called the corresponding eigenvector, 
i.e. if 

AX = /IX 
AX-/IX = 0 
AX-XIX = 0 
(A-AI)X = 0 

To find the eigenvalues we have to solve the equation 
\A-XI\ = 0 


Example 1 


Find the eigenvalues of A = 


1 

1 


Solution 


0 

-2 


A = 


1 

1 


0 

-2 



"1 0 ' 


"1 O' 


"1 0' 


"A 

o' 


o 

"X 

1 

II 

1 

1 -2 

-X 

0 1 

— 

1 -2 


0 

x\ 

— 

1 -2-X 


A- Xl\ = {\- X)(-2 - X) 


Solving | A - XI\ = 0 , we get 

(1 - X)(-2 -X) = 0 
So, either (1 - X) = 0 or (-2 - X) = 0 

so, X = 1 X = -2 are the eigenvalues of A. 


Power Method 


Let a matrix A is diagonalizable, with n linearly independent eigenvectors, 
V^,V 2 ,.. .V and corresponding eigenvalues A^,A 2 ,.. X such that 



> 


> 


1 


Since {v,,v 2 ,. 


,.v n ) is a basis fori?", any vector x can be written as 

x = c ] v l +c 2 v 2 +... + c n v n 
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Multiplying both sides by A, we get 
Ax = A(c 1 v l +c 2 v 2 +... + c n v n ) 

= Aw ) + A(c 2 v 2 ) +... -A(c n v n ) 

= q (A q) + c 2 (Av 2 ) +.. ,c n (Av n ) 

= q (A l v l ) + c 2 (A 2 v 2 ) + .. ,c n (A n v n ) 

Again multiplying by A and simplifying as above, we get 


A 2 x = c 1 (A, 2 v ,) + c 2 (T 2 2 v 2 ) + ...c n (A n \) 
Continuing this process we get 


A k x = q (A l k v l ) + c 2 (A 2 a v 2 ) + ...c n (A n k v n ) 

A k x = c 1 (A I ) k Vj + c 2 (A 2 ) A v 2 ) +...q, (A) k v n (k = 1,2,...) 


1 \k Ak __, „ 


(f-r = C V 1 + c 2 (q 1 )* v 2 ) + • • -C (qr)* v„ (* = 1 , 2 ,...) 

A A A 


(/Ij) k A k x —> CjVj 

ask ^ co 


Since 


1 


>k >...> 




Thus for large k, a scalar multiple of A k x detennines almost the same direction as the 
eigenvector qv t . Since, positive scalar multiples do not change the direction of the 

vector, A k x itself points almost in the same direction as v, or-q, providedq ^ 0. 


Procedure for finding the Eigenvalues and Eigenvectors 
Power Method 


To compute the largest eigenvalue and the corresponding eigenvector of the system 

A(X) = A(X) 

Where A is a real, symmetric or un-symmetric matrix, the power method is widely used 
in practice. 

Procedure 


Step 1: Choose the initial vector such that the largest element is unity. 

( 0 ) 

Step 2: The normalized vector v is pre-multiplied by the matrix A. 

Step 3: The resultant vector is again normalized. 

Step 4: This process of iteration is continued and the new normalized vector is repeatedly 
pre-multiplied by the matrix A until the required accuracy is obtained. 

At this point, the result looks like 

=[A]v (k - l) = q y k) 
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Here, is the desired largest eigenvalue and v< ' is the corresponding eigenvector. 

Example 2 


Let A = 


1.8 .8 


v 0.2 1 . 2 , 




vly 


andX = 


f-0.5^ 

v 1 j 


. Then A has Eigenvalues 2 and 1, and the 


eigenspace for \ = 2 is the line through 0 andvj. For k=0, ..8, compute A k x and 
construct the line through 0 and A k x . What happens as k increases? 

Solution 


The first three calculations are 


"1.8 

0.8" 

-0.5] 


~-0.1~ 

0.2 

1.2 

1 


1.1 


Ax = 

A 2 x = A(Ax) = 

A 3 x = A(A 2 x) = 

Analogous calculations complete Table 1. 
Table 1 Iterates of a Vector 


1.8 

0.8] 

-o.r 


"0.7" 

0.2 

\.2j 

i.i 


L3 

"1.8 

0.8' 

ro.7" 


"2.3" 

0.2 

1.2 

|_L3_ 


L7_ 


k 

0 

1 

2 

3 

4 

5 

6 

7 

8 

A k x 


"-.5] 

1 



~-0.l' 

1.1 



1 1 

O -H* 

1_1 



1 1 

fiJ 
ijj 

i_i 



~5.5~ 

_2.5_ 



11.9 

4.1 



~24.7~ 

_ 7 - 3 _ 



50.3" 

_!3.7_ 



101.5 

26 -5 _ 



The vectors x,Ax,A 2 x,...,A k x are shown in Fig. 1. The other vectors are growing too 
long to display. However, line segments are drawn showing the directions of those 
vectors. In fact, the directions of the vectors are what we really want to see, not the 
vectors themselves. The lines seem to be approaching the line representing the 
eigenspaces spanned by vi. More precisely, the angle between the line (subspace) 
determined by A k x and the line (eigenspaces) determined by v l goes to zero as k —» oo. 
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y J 

Figure 1 Directions determined by x, Ax, A x, ..A x 

The vectors (A, y )~ k A k x in (3) are scaled to make them converge toqvj, provided^ ^ 0. 
We cannot scale A k x in this way because we do not know \ but we can scale each A k x 
to make its largest entry 1. It turns out that the resulting sequence {x k } will converge to a 
multiple of v, whose largest entry is 1. Figure 2 shows the scaled sequence for Example 
1. The eigenvalue \ can be estimated from the sequence \x k ) too. When x k is close to an 
eigenvector for \ the vector Ax k is close to \x k with each entry in Ax k approximately 
\ times the corresponding entry in x k . Because the largest entry in x k is 1 the largest 
entry in Ax k is close to \ . 



Figure 2 Scaled multiples of x,Ax,A 7 x,...,A 7 x . 
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Steps for finding the eigenvalue and the eigenvector 

1. Select an initial vector x 0 whose largest entry is 1. 

2. Fork=0,l,.... 

a. Compute Ax k 

b. Let ju k be an entry in Ax k whose absolute value is as large as possible. 

c. Compute x k+l = (—) Aiy 

A* 

3. For almost all choices ofx 0 , the sequence {ju k } approaches the dominant 
eigenvalue, and the sequence {x k } approaches a dominant eigenvector. 


6 5 
1 2 


with x 0 = 


v 1 y 


. Stop when k = 5 , and estimate the 


Example 3 

Apply the power method to A = 
dominant eigenvalue and a corresponding eigenvector of A. 

Solution 

To begin, we calculate Ax 0 and identify the largest entry ju 0 in Ax 0 . 

Mo = 5 

Scale Alx 0 by 1/ fi 0 to get x,, compute Ax x and identify the largest entry in Ax x 


Ax 0 = 


1- 

so 

"O' 


'5' 

1 2 

1 


2 


Ao 


1 

"5l 


V 

5 

2 


.4 


AXj = 


"6 5" 

"1” 


" 8 " 

1 2 

.4 


L8 


A =8 


Scale Ax x by 1///, to get X 2 , compute Ax 2 , and identify the largest entry in Ax 2 . 


/u 2 =7.125 


1 

1 

" 8 ' 


1 

x, = — Ax, 



= 


Z 1 

A, 

~8 

i.8_ 


.225 


Ax 2 = 


"6 5" 

1 


"7.125" 

1 2 

.225_ 


1 .450 


Scale Ax 2 by 1/ ju 2 to get x 3 , and so on. The results of MATLAB calculations for the first 
five iterations are arranged in the Table 2. 

The evidence from the Table 2 strongly suggests that { x k } approaches (1, .2) and {//, } 
approaches 7. If so, then (1, .2) is an eigenvector and 7 is the dominant eigenvalue. This 


is easily verified by computing 


A = 


'll 


"6 5" 

'll 


" 7 " 

_ 7 

V 

.2 


1 2 

.2 


1.4 

— / 

.2 
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TABLE 2 The Power Method for Example 2 


k 

0 

1 

2 

3 

4 

5 

x k 


“O' 

_1_ 



"l" 

.4 



1 

.225j 


1 

_.2035J 


1 

_.2005j 


1 

_.20007j 

Ax k 


“5“ 

2 



1 1 
•3 oo 

oo 



“7.125] 
1.450 J 


“7.0175] 
1.4070J 


“7.0025] 

1.4010j 


“7.00036] 

1.40014J 

/4 

5 

8 

7.125 

7.0175 

7.0025 

7.00036 


Example 4 

Find the first three iterations of the power method applied on the following matrices 


“1 

1 

0“ 


“0“ 

2 

4 

2 

use x 0 = 

0 

0 

1 

2 


1 


Solution 


1 Iteration 


u x = Ax 0 - 


“1 

1 

0“ 

“ 0 “ 


“0 + 0 + 0“ 


“ 0 “ 

2 

4 

2 

0 

= 

0 + 0 + 2 

= 

2 

0 

1 

2 

1 


0 + 0 + 2 


2 


Now, we normalize the resultant vector to get 


= q x x x 



“ 0 “ 


“ 0 “ 

u \ = 

2 

= 2 

1 


2 


1 


2 nd Iteration 


u 2 = Axj = 


“1 

1 

0“ 

“ 0 “ 


“0+1+0“ 


“f 

2 

4 

2 

1 

= 

0 + 4 + 2 

= 

6 

0 

1 

2 

1 


0 + 1 + 2 


3 


Now, we nonnalize the resultant vector to get 
1 


u 2 = 


= 6 


= q 2 x 2 
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3rd Iteration 


Ax, = 

"1 1 0 " 

2 4 2 

“ll 

6 

1 


- + 1 + 0 
6 

—+ 4 + 1 


“ 7 “ 

6 

16 

Z 

0 1 2 

1 

_2_ 


3 

0 + 1 + 1 


3 

2 


Now, we normalize the resultant vector to get 


u 


3 


“ 7 ” 





“ 7 “ 

6 

16 

_ 16 

32 



1 

3 


3 

2 





_ 8 _ 


= q,x 3 


Hence the largest eigenvalue after 


3 iterations is —. 

3 


The corresponding eigenvector is 


1 _ 

32 

1 

3 

8 


The inverse power method 

1. Select an initial estimate sufficiently close to X . 

2. Select an initial vector x 0 whose large entry is 1. 

3. For k=0, 1, 2,... 

Solve (A - al)y k = x k . 

Let /u k be an entry in y k whose absolute value is as large as possible. 

Compute v k = a + (—). 

/4 

Compute x k+l = (— )y k 

4. For almost all the choice ofx 0 , the sequence {yj approaches the 
eigenvalue X of A, and the sequence \x k ) approaches a corresponding eigenvector. 
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Example 5 


It is not uncommon in some applications to need to know the smallest eigenvalue of a 
matrix A and to have at hand rough estimates of the eigenvalues. Suppose 21, 3.3, and 
1.9 are estimates for the eigenvalues of the matrix A below. Find the smallest 
eigenvalue, accurate to six decimal places. 


A = 


10 

-8 

-4 


-8 

13 

5 


-4 

4 

4 


Solution 


The two smallest eigenvalues seem close together, so we use the inverse power method 
for A — 1 .91 . Results of a MATLAB calculation are shown in Table 3. Here x 0 was 

chosen arbitrarily, y k = (A - 1 .91) 1 x k , ju k is the largest entry in y k , v k = 1.9 +1/ /u k , and 

*k + i =( 1 M)t a - 

Table 3: The Inverse Power Method 


K 

0 

1 

2 

3 

4 

x 0 


'll 

1 

1 



“0.5736“ 

0.0646 

1 



“0.5054“ 

0.0045 

1 



“0.5004“ 

0.0003 

1 



“0.50003“ 

0.00002 

1 


y k 


"4.45" 

0.50 

7.76 



“5.0131“ 

0.0442 

9.9197 



“5.0012“ 

0.0031 

9.9949 



“5.0001“ 

0.0002 

9.9996 



“5.000006 

0.000015 

9.999975 


M k 

7.76 

9.9197 

9.9949 

9.9996 

9.999975 

v k 

2.03 

2.0008 

2.00005 

2.000004 

2.0000002 


Therefore, we can say that eigenvalue is 2 from the matrix 



-8 

13 

5 


-4 

4 

4 
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Exercise 


How can you tell that if a given vector X is a good approximation to an eigenvector of a 
matrix A : if it is, how would you estimate the corresponding eigenvalue? Experiment 
with 



“5 

8 

4] 


" 1.0 “ 

A = 

8 

3 

-1 

And X = 

-4.3 


4 

-1 

2 


8.1 
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Lecture 36 
Revision 


Revision of Previous Lectures 
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Lecture 37 
Revision 


Revision of Previous Lectures 
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Lecture 38 
Inner Product 

If u and v are vectors inf?" , then we regard u and vas n x 1 matrices. The transpose 
u is a I x n matrix, and the matrix product u'v is a lxl matrix which we write as a 
single real number (a scalar) without brackets. 

The number u'v is called the inner product of a and v. And often is written as u.v 
This inner product is also referred to as a dot product. 



u \ 


v i 


u 2 


V 2 

If u = 


and v = 



_ U n\ 


_ V »_ 


Then the inner product of u and v is 


• • • U 


n 


= MjVj +U 2 V 2 +... + U n V n 


Example 1 



“ 2] 


~ 3~ 

Compute u.v and v.u when u = 

-5 

and v = 

2 


-1 


-3 


Solution 



' 2 ] 


‘ 3" 

u = 

-5 

and v = 

2 


-1 


-3 

u' = 

[ 2 _ 

- 5 - 1 ] 



u.v = u'v = [2 -5 -l] 


3 

2 

-3 


= 6-10 + 3 = -1 


2(3) + ( 5)(2) + (—1)(—3) 
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v' =[3 2 -3] 
v.u = v'u = [3 2 



= 6-10 + 3 = -1 


3(2) + (2)(—5) + (—3)( 1) 


Theorem 

Lett/, v and w be vectors in R" , and let c be a scalar. Then 

a. u.v = v.u 

b. (u + v).w = u.w+v.w 

c. (cu).v = c(u.v) = u.(cv) 

d. u.u > 0 and u.u = 0 if and only if u = 0 

Observation 


(C,M, +C 2 U 2 +....C p U p ).W 

= C) (u J .w) + c 2 ( u 2 .w) +.. .c (u p .w) 

Length or Norm 

The length or Nonn of v is the nonnegative scalar ||v|| defined by 
||v|| = = -^/vj 2 +Vj 2 + ... + v ;i 2 

||v||- =v.v 

Note: For any scalar c, ||cv|| = |c|||v|| 

Unit vector 

A vector whose length is 1 is called a unit vector. If we divide a non-zero vector 
vby its length ||v||, we obtain a unit vector u as 

v 

u = jj-| 

The length of u is ||w|| = jT| ll v ll = 1 


Definition 

The process of creating the unit vector u from v is sometimes called normalizing v, 
and we say that u is in the same direction asv. In this case “w ” is called the 
normalized vector. 

Example 2 

Let V = (1,2,2,0) ini? 4 . Find a unit vector u in the same direction as v. 

Solution 

The length of v is given by 
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||v|| = v/v/V = yfi + V 2 2 + V 3 2 + 

So, 

||v|| = Vl 2 +2 2 +2 2 +0 2 = Vl + 4 + 4 + 0 = V9 = 3 
The unit vector u in the direction of v is given as 


1 1 

u = —v = — 
V 3 


1 

2 

2 

0 


3 

2 _ 

1 

2 

3 

0 


To check that 



«|| = 1 

r l ~ 0 —2 i 2 o T 

(-) 2 + ( y ) 2 +(-) 2 + ( °) 2 


fl 4 4 

. —l-1— 

19 9 9 


+ 0=1 


Example 3 

2 

Let W be the subspace off? 2 spanned byX = (— ,1) . Find a unit vector Z that is a basis 
for W. 


Solution 

fE consists of all multiples of x, as in Fig. 2(a). Any nonzero vector in W is a basis 
for W. To simplify the calculation, x is scaled to eliminate fractions. That is, multiply 
x by 3 to get 


2 



Now compute y = 2 2 +3 2 = 13, y = Vl3, and normalize y to get 


1 

"2l 


"2/Vl3" 

Z “Vl3 

3 


_3/Vi3_ 


See Fig. 2(b). Another unit vector is (-2/\/l3 ,-3/Vl3). 
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(a) 

*1 



(b) 

Figure 2 Normalizing a vector to produce a unit vector. 


Definition 

For u and v vectors ini?", the distance between a andv, written as dist (u , v), is the 
length of the vector m -v . That is 
distill,v ) = ||i/ — v|| 

Example 4 

Compute the distance between the vectors u = (7, 1) and v= (3, 2) 

Solution 


“7“ 


"3l 


1 

^1 

1 

LO 
_1 


“ 4“ 

1 


2 


1-2 


-1 


dist (w , v) = - v|| = V( 4 ) 2 + ("I) 2 =Vl6 + l = Vl7 


Law of Parallelogram of vectors 

The vectors, u ,vand u-v are shown in the fig. below. When the vector u-v is 
added to v , the result isw . Notice that the parallelogram in the fig. below shows that 
the distance from u to v is the same as the distance of u-v too . 
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Example 5 

If u = (i/j, u 2 , u 3 ) and v = (vj, v 2 , v 3 ), then 
dist(u,v ) = ||w — v|| = ^(w -v).(m -v) 

= VOi - Vj ) 2 + (m 2 - V 2 ) 2 + («3 - V 3 ) 2 

Definition 

Two vectors in and v in f?" are orthogonal (to each other) if u.v = 0 

Note 

The zero vector is orthogonal to every vector in R" because O.V = 0 for all v inf?". 

The Pythagorean Theorem 

Two vectors u and v are orthogonal if and only if w + v = w + v 

Orthogonal Complements 

The set of all vectors z that are orthogonal to w in W is called the orthogonal 
complement of W and is denoted by W 1 

Example 6 

Let IE be a plane through the origin in R 3 , and let L be the line through the origin 
and perpendicular to W. If z and w are nonzero, z is on L, and w is in W, then the line 
segment from 0 to z is perpendicular to the line segment from 0 to w; that is, z . w = 0. 
So each vector on L is orthogonal to every w in W. In fact, L consists of all vectors 
that are orthogonal to the w’s in W, and W consists of all vectors orthogonal to the z’s 
in L. That is, 

L = W L and W = L 1 
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Remarks 


The following two facts about W 1 , with W a subspace of R", are needed later in the 
segment. 

(1) A vector x is in W 1 if and only if x is orthogonal to every vector in a set that 
spans W. 

(2) W L is a subspace of R n . 


Theorem 3 

Let A be m x n matrix. Then the orthogonal complement of the row space of A is the 
null space of A, and the orthogonal complement of the column space of A is the null 
space of A t : (Row A ) ± = Nul A, (Col A) 1 = Nul A T 

Proof 

The row-column rule for computing Ax shows that if x is in Nul A, then x is 
orthogonal to each row of ,4 (with the rows treated as vectors in /?"). Since the rows of 
A span the row space, x is orthogonal to Row A. Conversely, if x is orthogonal to 
Row A, then x is certainly orthogonal to each row of A, and hence Ax = 0. This 
proves the first statement. The second statement follows from the first by replacing A 
with A t and using the fact that Col A = Row A 1 . 


Angles in and R - 

If u and v are nonzero vectors in either R 2 or R 3 , then there is a nice connection 
between their inner product and the angle <9 between the two line segments from the 
origin to the points identified with u and v. The fonnula is 

uv = ||k||||v||cos«9 (2) 

To verify this formula for vectors in R 2 , consider the triangle shown in Fig. 7, with 
sides of length ||h||,||v||, and |//-v||. By the law of cosines, 

iii/—v i r=ii«ir+iivir—2 ii«h iivii cos ^ 


which can be rearranged to produce 


||//||||v||cos«9 = — 

|m| + v| 2 -|l/-v| 

II 1111 11 2 

-. 11 - 
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Example 7 

Find the angle between the vectors u = (1, -1,2), v = (2,1,0) 


Solution 

u.v = (1)(2) + (-1)(1) + (2)(0) = 2 -1 + 0 = 1 
And 

\\u\\ = V(l) 2 +(-l) 2 +(2) 2 = Vl + 1 + 4 = V6 
|v| = V(2) 2 +(l) 2 +(0) 2 = V4TT = Vs 


Angle between the two vectors is given by 

MV 

cos# = . 

M V 

Putting the values, we get 


cos# = 


cos# = 


<9 = cos 


1 

V#V5 

1 

^30 


V30 


Exercises 


1 

V30 


79.48° 


Q .i 

V 


"-3" 

Compute u.v and v.u when u = 

5 

and v = 

1 


3 


5 


Letv = (2,1,0,3) ini? 4 . Find a unit vector u in the direction opposite to that ofv. 

Q.3 

Let W be the subspace ofi? 3 spanned byX 
basis for W. 

Q.4 

Compute the distance between the vectors u = (1, 5, 7) and v= (2, 3, 5). 

Q.5 

Find the angle between the vectors u = (2,1,3), v = (1,0,2). 


= Find a unit vector Z that is a 

2 2 2 


©Virtual University Of Pakistan 


459 




39-Orthogonal and Orthonormal Sets 


VU 


Lecture No.39 

Orthogonal and Orthonormal sets 


Objectives 

The objectives of the lecture are to learn about: 

• Orthogonal Set. 

• Orthogonal Basis. 

• Unique representation of a vector as a linear combination of Basis vectors. 

• Orthogonal Projection. 

• Decomposition of a vector into sum of two vectors. 

• Orthonormal Set. 

• Orthonormal Basis. 

• Some examples to verify the definitions and the statements of the theorems. 

Orthogonal Set 

Let S = {u v u 2 ,...,u p }be the set of non-zero vectors in R n , is said to be an orthogonal set 

if all vectors in S are mutually orthogonal. That is 
O <£ S and u i .u j =o\/i ^ j,i,j = \,2,...,p. 


Example 

Show that S = {u l ,u 2 ,u 3 ) is an orthogonal set. Where 


u ] = 

"3l 

1 

,u 2 = 

"-1" 

2 


1 


1 


and 


-1 

~ 2 ~ 

-2 

7 

2 


Solution 

To show that S is orthogonal, we show that each vector in S is orthogonal to other. That 
is 

u.Mj =oVi*j,i,j = 1,2,3. 

For i = \,j = 2 



Which implies u t is orthogonal to n 7 . 
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For / = \,j = 3 



=- 2 + — 

2 2 

= 0 . 


Which implies u x is orthogonal to u 3 . 
For i = 2,j = 3 



=—4 + — 
2 2 

= 0 . 


Which implies u 2 is orthogonal to n 3 . 

Thus S = {wj,m 2 ,m 3 } is an orthogonal set. 

Theorem 

Suppose that S = {u l ,u 2 ,...,u p } is an orthogonal set of non-zero vectors in R" and 
W = Span \u x ,u 2 ,...,u p }. Then S is linearly independent set and a basis for W. 

Proof 

Suppose 

0 = C x u J +C 2 W 2 + ... + C M . 

Where c v c 2 ...,c are scalars. 

Mj . 0 = Uj . (Cjt/j + c 2 u 2 +... + c u ) 

0 =Mj + .(C 2 W 2 ) + ... + Mj -(C p U p ) 

= C) (u : ,U 1 ) + C 2 (U 1 ,U 2 ) + ... + C (Mj .M p ) 

= Cjtf/j . M, ) 
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Since S is orthogonal set, so, u x .u 2 + ... + u x .u p = 0 but u x ,u x > 0. 

Thereforeq = 0 . Similarly, it can be shown that c 2 = c 3 =... = c p = 0 

Therefore by definition S-iu x ,u 2 ,...,u p j is linearly independent set and by definition of 

basis is a basis for subspace W. 


Example 


If S = {u v u 2 } is an orthogonal set of non-zero vector in R . Show that S is linearly 
independent set. Where 


ll \ 


and u 2 = 


-1 

3 


Solution 

To show that S = {u l ,u 2 ] is linearly independent set, we show that the following vector 

equation 
CjMj +c 2 u 2 = 0. 

has only the trivial solution, i.e. c x =c 2 = 0. 

CjM, + c 2 u 2 = 0 


"3" 


'-f 


"o" 


+ c 2 


= 


_1_ 


_ 3_ 


_0_ 


3c, 

+ 

“ C 2 


"o" 

_ c i . 


m 


_0_ 


3c, - c, = 0 
c, + 3c 2 = 0 

Solve them simultaneously, gives 

C\—c 2 — 0 . 

Therefore if S is an orthogonal set then it is linearly independent. 

Orthogonal basis 

Let S=\u x ,u 2 ,...,u p } be a basis for a subspace W of R ", is also an orthogonal basis if S 
is an orthogonal set. 

Theorem 

If £={«,,« 2 ,...,n /) jis an orthogonal basis for a subspace W off?". Then each vin W can 
be uniquely expressed as a linear combination of u x ,u 2 ,...,u p . That is 
y = c x u x +c 2 u 2 + ... + c u . 
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Where 

y.Uj 

/> — __ 


Proof 


y. u x = (CjMj + C 2 U 2 + ... + CU ). Mj 

= (CjWJ.Mj +(c 2 U 2 ). M[ + ... + (c M ).!<, 

= Cj(m, .Mj) + c 2 (m, ,w 2 ) +... + c (u i -ll p ) 

= Cj(Wj .«,). 

Since S' is orthogonal set, so, m i .m 2 + ...+m,.m =0 but .t/ 1 > 0. 
Hence 


c, = ^ ' — and similarly c 9 = ^' Ul <? 

Mj . Mj w 2 . w 2 






Example 

The set <S , = {n 1 ,M 2 ,w 3 } as in first example is an orthogonal basis for R 3 . Express y as a 
linear combination of the vectors in S. Where 
v=[6 1 -8f 


Solution 

We want to write 

y =c l u l +c 2 u 2 + c 3 w 3 

Where c x ,c 2 and c 3 are to be determined. 
By the above theorem 
_ v.», 

Wj . Mj 



' 6" 


3" 



1 


1 



-8 


1 

11 


3" 


3" 

“IT 


1 


1 



1 


1 
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Hence 

y =Mj -2 u 2 -2u 3 . 


Example 


The set S = [u x ,u 2 , w 3 | is an orthogonal basis fori?'. Write y as a linear combination of 
the vectors in S. Where 



and u 3 



Solution 


We want to write 
y =c 1 M 1 +c 2 u 2 +c 3 n 3 
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Where c x ,c 2 and c 3 are to be determined. 
By the above theorem 



And 


" 3 i r °" 

7 . 0 



Hence 

y =-2mj +5m 2 + 4m 3 . 


Exercise 

The set S = [u x ,u 2 ,u^ is an orthogonal basis fori?'. Write y as a linear combination of 
the vectors in S. where 
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An Orthogonal Projection (Decomposition of a vector into the sum of two vectors 

Decomposition of a non- zero vector y e R n into the sum of two vectors in such a way, 
one is multiple of u e R" and the other orthogonal to u . That is 

y = y A +z 

Where y A — OtU for some scalar CC and z is orthogonal tow . 



In the above figure a vector y is decomposed into two vectors z = y - y A and y A = au. 
Clearly it can be seen that z = y — y A is orthogonal to u and v A = au is a multiple ofw . 
Since z = y-y' is orthogonal to u. 

Therefore 
z.w = 0 

(y-y A ).w = 0 
( y-au).u = 0 
y.u-a(u.u) = 0 
v.u 

=> a = 1 — 
u.u 

And 

z = _y-_y A 
y.u 

=y~- — u 
u.u 
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Hence 

V tl 

v A = u , which is an orthogonal projection of y onto it. 

u. u 

And 

*= y-y A 

v. w is a component of v 

=y-~—u 

u.u 


Example 


Let y = 


and u = 


Find the orthogonal projection of y onto u. Then write y as a sum of two orthogonal 
vectors, one in span {u} and one orthogonal to u. 


Solution 


Compute 


y u = 


uu = 


= 40 

= 20 


The orthogonal projection of y onto h is y = 


y-u 40 

- u = —u = 2 

"4l 

= 

"8" 

u ■ u 20 

2 


4 


and the 


component ofy orthogonal to u is y — y = 

The sum of these two vectors isy. That is, 


“7“ 


"8” 



6 


4 


2 


“7“ 


"8” 


"-1" 


= 


+ 


6 


4 


2 


y y (y-y) 


This decomposition of y is illustrated in Fig. 3. Note: If the calculations above are 
correct, then {y,y-y} will be an orthogonal set. As a check, compute 


J-(J-J) 



-8 + 8 = 0 
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Example 

Find the distance in figure below from y to L. 



Solution 

The distance from y to L is the length of the perpendicular line segment from y to the 

/v 

orthogonal projection JV 7 . 

The length equals the length of y — y. 

This distance is 

||v-v|| = V(-i ) 2 + 2 2 =V5 
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Example 

Decompose y = (-3,-4) into two vectors y and z, where y is a multiple of u = 
(-3,1) and Z is orthogonal to U. Also prove that y A .z =0 

Solution 


It is very much clear that /is an orthogonal projection of y onto ll and it is calculated by 
applying the following formula 
a y-u 

y =— u 

u.u 





Now 
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3" 


3“ 

2 


2 

1 


9 

_ 2 J 


2_ 


9 9 
~ 4 4 
= 0 


Therefore, y A is orthogonal to Z 

Exercise 

Find the orthogonal projection of a vector y = (-3, 2) onto u = (2, 1). Also prove that 
y=y* +z, where / a multiple of U and z is is an orthogonal to U. 

Orthonormal Set 

Let S = ^u l ,u 2 ,...,u p ]be the set of non-zero vectors in R", is said to be an orthonormal set 
if S is an orthogonal set of unit vectors. 

Example 

Show that S = { u { , u 2 , w 3 } is an orthonormal set. Where 


" 2 ' 


1 " 

VI 


" O' 


VI 

0 

, u 2 

-1 

and w 3 = 

0 

-1 


0 


2 

Lvi_ 


_ VI 


Solution 


To show that S is an orthonormal set, we show that it is an orthogonal set of unit 
vectors. 

It can be easily prove that S is an orthogonal set because 
u.Mj = 0Vi*j,i,j = 1,2,3. 

Furthermore 
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" 2 ' 


1 " 

J~5 


~ O' 


7F 

0 

,u 2 = 

-1 

u 2 = 

0 

-1 


0 


2 

L^j 


_7?_ 


' 2 ' 


' 2 ' 



V? 

0 


0 

-1 


-1 

Lvsj 


[yfsi 


=-+ 0 +- 
5 5 

= 1 



" O' 


" 0" 

• ^2 — 

-1 


-1 


0 


0 


= 0 + 1+0 
= 1 


And 


1 " 


1 ' 

7F 


7? 

0 


0 

2 


2 

_7?_ 


_7F 


_ 1 4 
~5 5 

= 1 

Hence 

S = | u { , u 2 , u 3 } is an orthononnal set. 


Orthonormal basis 

Let S = j be a basis for a subspace W of R 11 , is also an orthonormal basis if 

S is an orthonormal set. 


Example 
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Show that S = [u x , u 2 , u 3 } is an orthonormal basis of R', where 


_ 


-ll 


_ 

3 

vn 


2 


-1 

v/66 

i 

vn 

, U-2 

Te 

i 

A J6 

and u 3 = 

-4 

y[66 

i 

[_ vn_ 



7 

V66 


Solution 

To show that S = \u v u 2 ,u^ is an orthonormal basis, it is sufficient to show that it is an 

orthogonal set of unit vectors. That is 
u,Mj = 0Vi*j,i,j = 1,2,3. 

And 

Uj • uj = 1V/ = j,i,j = 1,2,3. 

Clearly it can be seen that 
w, ,u 2 =0, 

Mj .w 3 = 0 

And 

u 2 .w 3 =0. 

Furthermore 
u ] ,u x =1, 

U 2 • ii 2 — 1 

And 

u 2 ,u 2 = 1. 

Hence S is an orthonormal basis of R 3 . 

Theorem 

A mxn matrix U has orthononnal columns if and only if U'U = I 

Proof 

Keep in mind that in an if and only if statement, one part depends on the other, 
so, each part is proved separately. That is, we consider one part and then prove the other 
part with the help of that assumed part. 

Before proving both sides of the statements, we have to do some extra work which is 
necessary for the better understanding. 

Let u l ,u 2 ,...,u m be the columns of U. Then U can be written in matrix form as 
U=[u l u 2 u 2 ... u m ] 

Taking transpose, it becomes 
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u* 


u 

u 

u 


t 

1 

t 

2 
t 
3 


u[ 



i 

11^2 ••• 

U \ U ,n 

u' 2 



u‘ 2 u J 

U 2 U 2 ^ 2^3 

■ W 2 U m 

t 

u 3 

[wj u 2 u 3 . . 

. u 1 = 

m J 

u[u l 

W 3 I /2 ^ 3^3 ••• 

u\u m 

u' 

m 



ll'll 

m 

. UlUr. ll'll , 

1 ml mi 

- 


As u .v = v' u 
Therefore 


Mj, 

,u ] 

u v 

U 2 

Wj .«3 

u 2 

.11 x 

u 2 

•« 2 

1/3 *^3 

u 3 

.11 1 

u 3 

,w 2 

^3 


U‘U = 


... u 2 .u m 


ll,„ .11. 


Now, we come to prove the theorem. 

First suppose that U' U = /, and we prove that columns of U are orthonormal. 
Since, we assume that 
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l/j . 11 ^ 2 Wj .W 3 ... 

1 ^ 21 ^ 2*^2 ^ 2*^3 *** ^2 


W 3 .Wj W 3 .W 2 W3.W3 ... w 3 .w m 


c/'c/= 


W.W, W„,.W, 

ml m2 mi 


... t/ .t/ 

m m 


10 0 ... 
0 10... 
0 0 1 ... 


0 

0 

0 


0 0 0 ... 1 


Clearly, it can be seen that 
u t .Uj= 0 for i ^ j i,j = 1 , 2 ,...m 

and 

u i .u, = 1 /or i = j i,j =1,2,...m 


Therefore, columns of U are orthonormal. 

Next suppose that the columns of U are orthononnal and we will show that £/' t/ = I. 
Since we assume that columns of U are orthonormal, so, we can write 
u r Uj= 0 fori ^ j i,j = 1 , 2 ,...m 

and 

ir.iij = 1 for i = j i,j= 1 , 2 ,...m 


U v ll x 

Mj ,U 2 

u ] .u 3 .. 

• U V U m 


1^2 »^2 

tt 2 .w 3 . 

.. u 2 .w m 


I /3 .W 2 

t/ 3 .M 3 . 

.. u r u m 


Hence, U‘U = 
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10 0 ... 0 

0 10 ... 0 

0 0 1 ... 0 


0 0 0 ... 1 


That is 
U‘U = I. 

Which is our required result. 


Exercise 

Prove that the following matrices have orthonormal columns using above theorem. 


( 1 ) 

( 2 ) 


1 1 

~2 

1 

2 


1 

-1 

-2 1 
2 2 

1 -2 


(3) 


cos 6 sin <9 

-sin <9 cos 0 


Solution (1) 
Let 


u-' 

s[2 


U‘ = 


1 


V2 


1 

1 

1 

1 


Then 


U‘U = 


l_ 1 
2 |_1 
"l 
0 


1 

-1 

1 

-1 


1 

-1 

0 

1 


= 7 



U‘U=I 

Therefore, by the above theorem, U has orthononnal columns. 


(2) And (3) are left for reader. 
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Theorem 

Let U be an m x n matrix with orthonormal columns, and let x and y be in R n . Then 

a) ||[/x|| = ||x|| 

b) (Ux).(Uy) = x.y 

c) (Ux).(Uy) = 0 iff x.y — 0 


Example 


"l/V2 

Let U =-^ 1/V2 

[° 

Verify that ||Lh|| = ||x|| 

Solution 

Notice that U has orthonormal columns and 



1/V2 1/V2 

_2/3 -2/3 


0 

1/3 


1/V2 

2/3 

1/V2 

-2/3 

0 

1/3 




"l/V2 
C/ x = ^ 1/V2 

[° 

|c/x|=V9+1+1 = vn 

iixii = V2+9 = vn 
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Lecture No.40 
Orthogonal Decomposition 


Objectives 

The objectives of the lecture are to learn about: 

• Orthogonal Decomposition Theorem. 

• Best Approximation Theorem. 

Orthogonal Projection 

The orthogonal projection of a point in R 2 onto a line through the origin has an important 
analogue in R" . 

That is given a vector Y and a subspace W in R", there is a vector y in W such that 

1) y is the unique vector in W for which y - y is orthogonal to W, and 

2) y is the unique vector in W closest to y. 


ry 


o 


y 


We observe that whenever a vector y is written as a linear combination of vectors 
u v u 2 ,...,u n in a basis of R", the terms in the sum for y can be grouped into two parts so 
that y can be written as y = z x +z 2 , where z, is a linear combination of some of the m. ’s, 
and z 0 is a linear combination of the rest of the u j 's . This idea is particularly useful when 
{u r u 2 ,...,u n } is an orthogonal basis. 

Example 1 

Let {h 1 ,k j ,...,m 5 } be an orthogonal basis for R 5 and let y = c l u 1 +c 2 u 2 +... + c 5 u 5 . 

Consider the subspace W= Spanjui, U 2 } and write y as the sum of a vector z, in W and a 
vector z 2 in W 1 . 


Solution 
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Write 

V = CjWj + c 2 u 2 + c 2 u 2 + c 4 u 4 + c 5 u 5 

Z 1 z 2 

where z x = c ] u ] +c 2 u 2 is in Span of {ui, 112 }, and z 2 = c 3 w 3 +c 4 u 4 +c 5 u 5 is in Span of { 113 , 
U 4 , Us}. 

To show that Z2 is in W L , it suffices to show that 7.2 is orthogonal to the vectors in the 
basis {uj, U 2 } for W. Using properties of the inner product, compute 

z 2 • Mj = (c 3 M 3 + C 4 U 4 + C 5 U 5 ) • Mj = c 3 w 3 • u l + c 4 u 4 • u ] + c 5 u 5 ■ u l =0 

since Hi is orthogonal to U 3 , u 4 , and Us, a similar calculation shows that z 2 u 2 = 0 
Thus, z 2 is in W L . 

Orthogonal decomposition theorem 

Let W be a subspace of R", then each y in R" can be written uniquely in the fonn 

y = y + z 

Where y A e W and z e W 1 

Furthermore, if {u x ,u 2 ,...,u p } is any orthogonal basis for W, then 

y.u, 

y A = CjHj + c 2 u 2 +... + c n u n , where c . = -—— 

Uj.Uj 

Proof 


z = y-y y 



Fig: Orthogonal projection of y on to W. 


Firstly, we show that y A <=W , z e W L . Then we will show that y = y A +z can be 
represented in a unique way. 

Suppose W is a subspace of R" and let jw 15 w 2 ,...,w p | be an orthogonal basis for 
W. 

y.u. 

As y A =CjHj +c 2 u 2 ...c u p where c. =J —— 
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Since \u x ,u 2 ,...,u p } is the basis for W and y A is written as a linear combination of 
these basis vectors. Therefore, by definition of basis y A eW. 

Now, we will show that z = y - y A e W L . For this it is sufficient to show that z _L u j for 
each j= 1 , 2 ,.../ 7 . 

Let Wj eW be an arbitrary vector. 

z • «i =(y~y A )- u i 

= y.u l -y A .u l 

= y.u l -(c l u l +c 2 u 2 +... + c p u p ).u l 
= y-U l ~C x (u x .U x )-C 2 (u 2 M 1 )-...c p (u p .Mj) 

= y.u l -Cj(Wj .t/|) where Uj .w, = 0 ,7 = 2,3,.../? 

T-Mj , . 

= T-M 1 -(Mj.Mj) 

M| .Mj 

= y.w 1 -y.w 1 
= 0 

Therefore, zIh,. 

Since tq is an arbitrary vector, therefore z_L m ; . for 7 = 1 , 2 ,.../?. 

Hence by definition of JF^zefF 1 

Now, we must show that y = y A + z is unique by contradiction. 

Let y=y A +z and y=y 1 A + z,, where y A , y, A e W and z,z, e IF" 1 , 

also z Zj and y A ^ y A . Since above representations for y are equal, that is 

A . A . 

y +z = yj +Zj 

_. A A 

=> y -Ti = z i~ z 

Let 

A A 

s = t ~y\ 

Then 
J = Zj - z 

Since W is a subspace, therefore, by closure property 
s = y A -y A eW 

Furthermore, W L is also a subspace, therefore by closure property 

s = z x -zew L 

Since 

s e W and s e W L . Therefore by definition s _L 5 

That is S. S=0 

Therefore 

s = y'-y[ =0 

_. A A 
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Also 
Zj = z 

This shows that representation is unique. 

Example 



' 2] 


'-2' 


'1 ' 

Let Mj = 

5 

? ^2 

1 

, and y = 

2 


-1 


1 


3 


Observe that {u v u 2 ) is an orthogonal basis for W=span {u r u 2 }, write y as the sum of a 
vector in W and a vector orthogonal to W. 

Solution 


Since v A e W, therefore v A can be written as following: 

y A = CjMj + c 2 u 2 


y-u i 

Mj .Mj 


Wj + 



1^2 *^2 



' 2] 


~-2 


' 2] 


"-2 

9 


3 

1 

9 


15 

1 


5 

+ — 


5 

H- 

30 

-1 

6 

1 

~30 

-1 

30 

1 

r- 

2/5 

1 

r-2‘ 

1 





2 

1/5 


10 

1 


y-y 


A 


" ll 


" -2/5] 


“7/5 

2 

- 

2 

= 

0 

3 


1/5 


14/5 


= 7/5 


1 

0 

2 


Above theorem ensures that y - y A is in W 1 . 

You can also verify by (y - y A ).u x = 0 and (y -y A ).u 2 = 0. 


The desired decomposition of y is 
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1 

-3 and u 2 
2 



2 

Decompose y = -2 into two vectors; one in W and one in W 1 . Also verify that these 
5 


two vectors are orthogonal. 

Solution 


Let v A € W and z = y - y A e W L . 

Since v A e W, therefore v A can be written as following: 
y A = CjMj + c 2 u 2 



“ 3" 
v A = -3 
3 


Now 

" 2i r 3" 

z = y-y A = -2-3 



Now we show that z 1 y A , i.e. z.y A = 0 
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"-U 


~ 3" 

A 

z.y = 

1 


-3 


2 


3 


= 0 


Therefore zly A . 

Exercise 



“ 1 “ 


“ 3" 

Let W=span{w 1 ,M 2 } , where u x = 

0 

-3 

and U-, = 

1 

1 


Write v = 


6 

-8 

12 


as a sum of two vectors; one in W and one in W 1 . Also verify that these 


two vectors are orthogonal. 


Best Approximation Theorem 


Let W is a finite dimensional subspace of an inner product space V and y is 
any vector in V. The best approximation to y from W is then Pr oj y w , i.e for every w (that 

is notPr oj y w ) in W, we have 

11^-Pro/dl <!!>'-HI- 


Example 


Let W=span{w 1 ,w 2 } , where u x = 

' 1 ' 

-3 

? ^2 

' 4" 

2 

and v = 

' 2" 

-2 


2 


1 


5 


. Then using above 


theorem, find the distance from y to W. 


Solution 

Using above theorem the distance from y to W is calculated using the following formula 
||y-Prq/^|| = |v-y A | 

Since, we have already calculated 


" 2] 


_ 3l 


"-1" 

-2 

- 

-3 

= 

1 

5 


3 


2 


So | v - y A | = y[6 

Example 
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The distance from a point y in R" to a subspace W is defined as the distance from y to the 
nearest point in W. 

Find the distance from y to W= span{u x ,u 2 } , where 



“-1] 


“ 5 “ 


“ 1 “ 

y = 

-5 

,M, = 

-2 

, u 2 — 

2 


10 


1 


-1 


By the Best Approximation Theorem, the distance from y to W is ||y-y||, where 
y = proj„y. Since, {ui, Ui\ is an orthogonal basis for W, we have 





“ 5 “ 


“ 1 “ 


-1 

15 

-21 

1 


7 



-8 

— u, + ■ 

- u 2 


-2 


2 

= 

30 

6 2 

~2 

1 

2 

-1 


4 


"-ll 


'-f 


" 0 " 

-5 

- 

-8 

= 

3 

10 


4 


6 


||v-v|| 2 =3 2 +6 2 =45 
The distance fromy to W is V45 = 3^5. 

Theorem 

If ji/j, u 2 ,...,u p | is an orthononnal basis for a subspace W of R ", then 
?roj w y=(y.u l )u l +(y.u 2 )u 2 +...(y.u p )u p 
If U = \u x u 2 ....u p ] 
then Pro/;’’ =UU T y \/yinR n 

Example 


Let Wj = 

"-7l 

1 

, u 2 

"-ll 

1 

» y = 

-9 

1 


4 


-2 


6 


and W= span{wj,M 2 } . Use the fact that Ui and U 2 are orthogonal to compute Pr oj w y . 

Solution 
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In this case,y happens to be a linear combination of Ui and U 2 . So y is in W. The closest 
point in W to y is y itself. 
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Lecture # 41 

Orthogonal basis, Gram-Schmidt Process, Orthonormal basis 

Example 1 

Let W= Span {xj, x?}, where 



" 3 ] 


V 

X, = 

6 

0 

and x 2 = 

2 

2 


Find the orthogonal basis {v/, for W. 

Solution 

Let P be a proj ection of X 2 on to Xi. The component of X 2 orthogonal to Xi is X 2 - P , 
which is in W as it is formed from X 2 and a multiple of x/. 

Let Vi=Xi and compute 

D X 2- V l 

v 2 =x 2 -P = X 2 -L = 

v t- v i 

Thus, {vi, V2} is an orthogonal set of nonzero vectors in W, dim W= 2 and {vi, V2} is a 
basis of W. 

Example 2 

For the given basis of a subspace W= Span {xi, x?}, 


"O' 


' 5 ' 

4 

and x 2 = 

6 

2 


-7 


Find the orthogonal basis {vj, for W. 

Solution 

Set Vi = Xj and compute 


1 

2 

15 

3 

6 


1 

O O 

1_ 

2 

45 

0 


2 
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10 

20 


0 

4 

2 


Thus, an orthogonal basis for W is 


0 

< 4 
2 


5 

4 > 
-8 


Theorem 


Given a basis {xj, ...,x p ) for a subspace W of R". Define 


Vi.Vj 


X,.V, X v V, 

V 3 = *3 - V 1 - V 2 


X p- V 1 X P - V 2 X p' V p- 1 

V P= X P - V 1 - V 2“- - V i 

. VfVl 


Then {v/, ...,v p } is an orthogonal basis for W. 
In addition 


Span {v i, Vk}= Span {xi,...,Xk} for 1 <k< p 

Example 3 

The following vectors {xi, x 2 , x 3 } are linearly independent 



'll 


"O' 


" 0 " 


1 


1 


0 

X, = 


,x, = 


,x, = 


1 

1 

7 z 

1 

7 J 

1 


1 


1 


1 


Construct an orthogonal basis for W by Gram-Schmidt Process. 
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Solution 

To construct orthogonal basis we have to perfonn the following steps. 
Step 1 Let vi=xi 

Step 2 


T X, .V. 

Letv, = x, — L v 1 

v i- v i 


Since 


"O' 


'll 


"-3/4" 


~-3~ 

1 

3 

1 


1/4 


1 

1 

4 

1 


1/4 


1 

1 


1 


_ 1/4 _ 


1 


Step 3 


v t- v i 


v [ + 


X V V 2 

V 2 .V 2 


'O' 



T 


"-3" 


' 0 ' 


0 


2 

l 

2 

1 


2/3 



— 

— 


H - 


= 



1 


4 

l 

12 

1 


2/3 


1 



l 


1 


_2/3_ 



"O' 


' 0 ' 


0 

0 


2/3 


-2/3 

1 


2/3 


1/3 

1 


_2/3_ 


_ 1/3 _ 


Thus, {vi, V 2 , Vi} is an orthogonal set of nonzero vectors in W. 


Example 4 

Find an orthogonal basis for the column space of the following matrix by Gram-Schmidt 
Process. 
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-16 6 
3-8 3 

1 -2 6 
1 -4 -3 


Solution 

Name the columns of the above matrix as xi, X 2 , X 3 and perform the Gram-Schmidt 
Process on these vectors. 


"-ll 


“6 ” 


“6 " 

3 


-8 


3 

1 

> x2 ~ 

-2 

, XT, ~ 

6 

1 


_-4_ 


_-3_ 


Set vi =xi 


* 2 - v t 

v 2 = X,--—- Vj 

v,.v, 


"6 " 


"-ll 


"3 " 

-8 


3 


1 


-(-3) 


= 


-2 


1 


1 

-4 


1 


-1 


V r Vi 1 v 2 ,v 2 


"6 ' 


'-f 


"3 “ 


~-l~ 

3 

1 

3 

5 

1 


-1 

6 

2 

1 

2 

1 


3 

_-3_ 


1 


-1 


-1 


Thus, orthogonal basis is 


'-f 


"3 " 


~-l" 

3 


1 


-1 

1 

9 

1 

9 

3 

1 


-1 


-1 
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Example 5 

Find the orthonormal basis of the subspace spanned by the following vectors. 


II 

X 

~3~ 

6 

,x 2 = 

1 

o o 

1 _ 


o 


K> 


Solution 


Since from example # 1, we have 



i 

U> 
_1 


o 

1 _ 

ii 

6 

II 

0 


1 

o 

1 _ 


i 

<N 

_i 


Orthonormal Basis 


1 

u x =-Vj 

v i 


1 

V45 


"3l 


l/yf5 

1 

, U 2 = II I, V 2 = 

"0" 

6 

= 

2/V5 

0 

0 


0 

Ihll 

1 


Example 6 

Find the orthonormal basis of the subspace spanned by the following vectors. 


Xl - 

_1 

and xi = 

~4~ 

-1 


1 


2 


Solution 


Firstly we find Vi and V 2 by Gram-Schmidt Process as 

*2- V l 

v 2 =x 2 = x 2 —— Vj 


Set vl =x i 

V, = 

" 4 1 

-1 

15 

_1 


' 4] 

-1 

1 

_1 

z 

2 

30 

1 


2 

2 

1 


©Virtual University Of Pakistan 


489 





41-Othogonal basis, Gram-Schmidt process. Orthonormal basis 


VU 


" 4 1 


1 


" 6 " 

-1 

- 

-5/2 

= 

3 

2 


1/2 


3 


Now ||v 2 1 = .— = — -j= and since |vj || = V30 
V54 3V6 


Thus, the orthonormal basis for W is 


r 2/v/30 


2/V6~| 

-5/V30 


1/V6 

1/V30 


1/V6 


Theorem 


If A is an m x n matrix with linearly independent columns, then A can be factored as 
A = QR, where Q is an in x n matrix whose columns form an orthonormal basis for Col A 
and R is an n x n upper triangular invertible matrix with positive entries on its diagonal. 


Example 7 


1 


Find a QR factorization of matrix A = 


-1 

-1 


1 

1 


2 

1 

4 

-4 

2 


5 

-4 

-3 

7 

1 


Solution 

Firstly find the orthonormal basis by applying Gram Schmidt process on the columns of 
A. We get the following matrix Q. 


l/v/5 

M2 

1/2 

—l/v/5 

0 

0 

-1/V5 

1/2 

1/2 

1/V5 

-1/2 

1/2 

1/ s/5 

1/2 

-1/2 
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Now R - Q J A 


s 

o 

o 


Verify that A=QR. 


-y/5 4^5 

6 -2 

0 4 


Theorem 

If {Mj, ..., u } is an orthonormal basis for a subspace W of R n ,then 

proj w y = (y ■u l )u l + (y ■ u 2 )u 2 + ■ ■ ■ + (y ■ u p )u p 

IfU= [m, u 2 ... u p ], 

then proj w y = UU T y V v in R" 


The Orthogonal Decomposition Theorem 

Let W be a subspace of R" Then eachy in R n can be written uniquely in the form 

y = y + z 

where w L is in W and z is in 

In fact, if {ui, u p } is any orthogonal basis of W, then 


„ v • u, 

y = - — w i+- 

Mj • Mj 


■ + - 


y ' 11 r 


u p' u p 


and z = y - y . The vector y is called the orthogonal projection of y onto W and is 
often written as prqj v , y. 


Best Approximation Theorem 

Let ILbe a subspace of R" , v is any vector in R n and y the orthogonal projection of y 
onto W. Then y is the closest point in W to y, in the sense that 

for all v in W distinct from y . 


||y - v|| < | v - v 


The vector v in this theorem is called the best approximation to y by elements of W. 
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Exercise 1 

Let W= Span {xl, x2}, where 

r 1/3 

x^ = 1 and x^ = 1/3 
|_lj [-2/3 

Construct an orthonormal basis for W. 

Exercise 2 

Find an orthogonal basis for the column space of the following matrix by Gram-Schmidt 
Process. 

"3-5 1" 

1 1 1 

-1 -5 -2 

3-7 8 _ 

Exercise 3 

Find a QR factorization of 

"1 3 5" 

-1 -3 1 

A= 0 2 3 

1 5 2 

1 5 8 
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Lecture # 42 
Least Square Solution 

Best Approximation Theorem 

Let Wbe a subspace of R n , y be any vector in r" and y the orthogonal projection ofy 
onto W. Then y is the closest point in W to y, in the sense that ||y - y\ \<\\y-4 

for all v in W distinct from y 

The vector y in this theorem is called the best approximation to v by elements of W. 

Least-squares solution 

The most important aspect of the least-squares problem is that no matter what “x ” we 
select, the vector Ax will necessarily be in the column space Col^L So we seek anx that 
makes Ax the closest point in Col A to b. Of course, if b happens to be in Col A, then b is 
Ax for some x and such an x is a “least-squares solution.” 

Solution of the General Least-Squares Problem 

Given A and b as above, apply the Best Approximation Theorem stated above to the 
subspace Col A. Let b = proj Col4 b 

Since b is in the column space of A, the equation Ax = b is consistent, and there is an x 
in R" such that 

Ax = b (1) 

Since b is the closest point in Col A to b, a vector x is a least-squares solution of Ax = b 
if and only if x satisfies Ax = b . Such an x in R" is a list of weights that will build b out 
of the columns of A 

Normal equations for x 

Suppose that x satisfies Ax = b. By the Orthogonal Decomposition Theorem the 
projection b has the property that b—b is orthogonal to Col A, so b-Ax is orthogonal 
to each column of A. If Uj is any column of A, then a ; . • ( b - Ax) = 0, and aj(h - Ax) = 0. 


Since each a J is a row of^ r , 


1 

II 

o 

(2) 

A T b-A r Ax = 0 


A t Ax=A T b 

(3) 


The matrix equation (3) represents a system of linear equations commonly referred to as 

the normal equations for x. 
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Since set of least-squares solutions is nonempty and any such x satisfies the normal 
equations. Conversely, suppose that x satisfies A' Ax = A T b. Then it satisfy that b- Ax 
is orthogonal to the rows of A T and hence is orthogonal to the columns of A. Since the 
columns of A span Col A, the vector b-Ax is orthogonal to all of Col A. Hence the 
equation b = Ax + (b- Ax) is a decomposition of b into the sum of a vector in Col A and 
a vector orthogonal to Col A. By the uniqueness of the orthogonal decomposition, Ax 
must be the orthogonal projection of b onto Col A. That is, Ax = b and x is a least- 
squares solution. 

Definition 


If A is m x n and b is in^' 1 , a least-squares solution of Ax = b is an lx in R n such that 

lib - Ax || < ||b - Ax || V x e R" 


Theorem 

The set of least-squares solutions of Ax = b coincides with the nonempty set of solutions 
of the normal equations 

A t Ax = A T b 

Example 1 

Find the least squares solution and its error from the following matrices, 



~4 

0 ] 


~2~ 

A = 

0 

2 

,b = 

0 


1 

1 


11 


Solution 

Firstly we find 






"4 

0 ” 




“4 

0 

f 




"17 f 

ll 

0 

2 

1 

0 

2 

= 

1 5 _ 





1 

1 




A T b = 



"2" 


"4 0 f 



19 


0 

= 


0 2 1 



11 


11 




and 


Then the equation A T Ax = A T b becomes 


"17 

f 



"19" 

1 

5 

_* 2 J 


11 
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Row operations can be used to solve this system, but since A 1 A is invertible and 2 x 2 , it 


is probably faster to compute (A T A) 1 


84 


5 

-1 


-1 

17 


1 

2 



'4 

O' 


~2~ 

Now again as A= 

0 

2 

, b = 

0 


1 

1 


11 




"4 

0 " 



4 


Ax = 



V 



Then 

0 

2 

2 

= 

4 




1 

1 

" 


3 



"2] 


"4" 


'-2 

Hence 

b-Ax = 

0 

- 

4 

= 

-4 



11 


3 


8 


1 

5 

-1 

19 

1 

84 


84 

-1 

17 

11 

“84 

I 

OO 

so 

_[ 



So \\b - Ax\\ = ^(-2) 2 + (-4) 2 + 8 2 = V84 

The least-squares error is V84. For any x in R 2 , the distance between b and the vector Ax 
is at least V84. 


Example 2 

Find the general least-squares solution of Ax = b in the form of a free variable with 


'1 

1 

0 

0 " 


~-3~ 

1 

1 

0 

0 


-1 

1 

0 

1 

0 

,b = 

0 

1 

0 

1 

0 

2 

1 

0 

0 

1 


5 

1 

0 

0 

1 


1 


Solution 
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Firstly we find, 








'1 

1 

0 

0 " 






"1 

1 

1 

1 

1 

1 " 

1 

1 

0 

0 


"6 

2 

2 

2 

1 

1 

0 

0 

0 

0 

1 

0 

1 

0 


2 

2 

0 

0 

0 

0 

1 

1 

0 

0 

1 

0 

1 

0 


2 

0 

2 

0 

0 

0 

0 

0 

1 

1 

1 

0 

0 

1 


2 

0 

0 

2 







1 

0 

0 

1 







and 








~-3~ 



"1 

1 

1 

1 

1 

f 

-1 


" 4 ' 

1 

1 

0 

0 

0 

0 

0 


-4 

0 

0 

1 

1 

0 

0 

2 


2 

0 

0 

0 

0 

1 

1 

5 

1 


6 


Then augmented matrix for A 7 Ax = A 7 b is 


"6 

2 

2 

2 

4 ' 


'1 

0 

0 

1 

3 " 

2 

2 

0 

0 

-4 


0 

1 

0 

-1 

-5 

2 

0 

2 

0 

2 


0 

0 

1 

-1 

-2 

2 

0 

0 

2 

6 _ 


0 

0 

0 

0 

0 


The general solution is x 3 = 3 - x 4 , x 2 = -5 + x 4 , x 3 = -2 + x 4 , and X 4 is free. 
So the general least-squares solution of Ax = b has the fonn 



" 3 ' 


~-l~ 


-5 


1 

x = 


+ x 4 



-2 

4 

1 


0 _ 


1 


Theorem 


The matrix A 7 A is invertible iff the columns of A are linearly independent. In this case, 
the equation Ax = b has only one least-squares solution x , and it is given by 


x = (A 7 A)~ 1 A r b 


Example 3 

Find the least squares solution to the following system of equations. 

° 1 

1 

-2 
4 

Solution 


A = 


1 -3 
7 1 

i r\ 


0 

4 


xl 

x 2 

x3 
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As 


Now 


A = 


2 1 
4 -3 
6 0 


7 1 
1 0 
4 5 






"2 

4 

6 ] 





"2 

1 

7 

f 

1 

-3 

0 


“55 

12 

45“ 

4 

-3 

1 

0 

7 

1 

4 

= 

12 

26 

28 

6 

0 

4 

5 





45 

28 

77 





1 

0 

5 J 











“0 “ 



A T b = 

“2 

1 

7 

f 

1 


-9“ 

4 

-3 

1 

0 

= 

-5 






-2 




6 

0 

4 

5 



12 






4 


— - 1 


As 


A t Ax = A T b 


“55 

12 

45“ 

xl 


-9“ 

12 

26 

28 

x2 

= 

-5 

45 

28 

77 

x3 


12 


xl 


“-.676“ 

x2 

= 

-.776 

x3 


.834 


Example 4 


Compute the least square error for the solution of the following equation 


2 4 
1 -3 
7 1 
1 0 




“0 ‘ 

x\ 


1 

x2 

,b = 

-2 

x3 



L “ 


4 


Solution 
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Ax = 


Ax = 


2 4 

1 -3 

7 1 

1 0 
0.548 ' 
1.652 
-2.172 
3.494 


6 

0 

4 

5 





'2 

4 

6 " 



xl 


1 

-3 

0 

"-.676" 


x2 

= 

7 

1 

4 

-.776 


x3 





.834 


“ _ 


1 

0 

5 _ 



As least square error 

||e|| = ||6-y4x|| 

is as small as possible, or in other words is smaller than all other possible choices of x. 


As 


“0 ' 


"0.548 ' 


'-0.548" 

1 


1.652 


0.652 

-2 


-2.172 


0.172 

4 


3.494 


1.494 


Thus, least square error is 


= e, + <E „ +G, +€„ 


|e|| = ||h - Ax\\ =^(-0.548) 2 + (0.652) 2 +(0.172) 2 +(1.494) 2 
=0.3003+0.4251+.02958+2.23=2.987 


Theorem 


Given an m x n matrix A with linearly independent columns.Let A = QR be a QR 
factorization of A ,then for each b in Rm, the equation Ax = b has a unique least-squares 
solution, given by 
x = R A Q T b 


Example 1 


© Virtual University Of Pakistan 


498 





42-Least Square Solution 


VU 


Find the least square solution for A = 


"1 

3 

5 “ 


"3 " 

1 

1 

0 

,b = 

5 

1 

1 

2 

7 

1 

3 

3_ 


_-3_ 


Solution 


First of all we find QR factorization of the given matrix A. For this we have to find out 
orthonormal basis for the column space of A by applying Gram-Schmidt Process, we get 
the matrix of orthonormal basis Q, 


0 = 


R = Q t A 


" 1/2 

1/2 

1 / 2 ' 

1/2 

- 1/2 

- 1/2 

1/2 

- 1/2 

1/2 

1/2 

1/2 

- 1/2 

1/2 

1/2 

1/2 

- 1/2 

- 1/2 

1/2 

- 1/2 

1/2 

- 1/2 


And 


"1 

3 

5 “ 









'2 

4 

5“ 

1 

1 

0 

= 

0 

2 

3 

1 

1 

2 


0 

0 

2 

1 

3 

3_ 






Then 



"1/2 

1/2 

1/2 

1/2 

Q T b = 

1/2 

-1/2 

-1/2 

1/2 


.1/2 

-1/2 

1/2 

-1/2 


3 

5 

7 

-3 


6 

-6 

4 


The least-squares solution x satisfies Rx = Q T b; that is, 

'2 4 5" 

0 2 3 

x 2 

— 

" 6 " 

-6 


0 0 2 

x 3 \ 


4 


This equation is solved easily and yields x = 


10 

-6 

2 


Example 2 


Find the least squares solution Rx - Q T b 


to the given matrices, 
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"3 

ll 


V 

A = 

6 

2 

,b = 

2 


0 

2 


3 


Solution 

First of all we find QR factorization of the given matrix A. Thus, 
have to make 

Orthonormal basis by applying Gram Schmidt process on the columns of A, 


Let Vi = Xi 


v 2 =x 2 —P = x 2 


Vj.Vi 


v, = 

V 

2 

15 

"31 

6 


1 

o o 

1_ 

1 

2 

45 

0 


2 


Thus 


Thus, the orthonormal basis are 


r 


T 


"O' 

' 



2 


0 


UK II IbllJ 


0 


1 



"1 

o' 



2 

0 

and Q T = 

'1 2 O' 



0 0 1 

0 

1 




Now R = Q T A — 


1 2 

0 0 



1 

2 

2 





V 

15 5" 

And Q T b = 

"1 2 0" 

2 

0 2 

0 0 1 


— 



3 


5 

3 


Thus, least squares solution of Rx = Q T b is 


"15 

5" 


“5“ 

0 

2 

x = 

_3_ 


"1 



x = 

_ L8 _ 




we 
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QExercise 1 

Find a least-squares solution of Ax = b for 


"1 -6 



1 -2 

,b = 

2 

1 

1 

1 

1 

7 _ 


6 


Exercise 2 


Find the least-squares solution and its error of Ax = b for 


"1 

3 

5“ 


' 3 " 

1 

1 

0 

,b = 

5 

1 

1 

2 

7 

1 

3 

3_ 


_-3_ 


Exercise 3 


Find the least squares solution Rx = Q T b to the given matrices, 



2 

r 


~-5~ 

A = 

-2 

0 

,b = 

8 


2 

3 


1 
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Inner Product Space 

In mathematics, an inner product space is a vector space with the additional structure 
called an inner product. This additional structure associates each pair of vectors in the 
space with a scalar quantity known as the inner product of the vectors. Inner products 
allow the rigorous introduction of intuitive geometrical notions such as the length of a 
vector or the angle between two vectors. They also provide the means of defining 
orthogonality between vectors (zero inner product). Inner product spaces generalize 
Euclidean spaces (in which the inner product is the dot product, also known as the scalar 
product) to vector spaces of any (possibly infinite) dimension, and are studied in 
functional analysis. 

Definition 

An inner product on a vector space V is a function that to each pair of vectors 
u and v associates a real number (u,v) and satisfies the following axioms, 

For all u,v,w in V and all scalars C: 

1) (u,v) = ( v,u) 

2) (n + v, w) = (u,w) + (v, w) 

3) (cu,v) = c(u,v) 

4) (u,u) > 0 and (u,u) =0 iff u = 0 

A vector space with an inner product is called inner product space. 


Example 1 

Fix any two positive numbers say 4 & 5 and for vectors u = iit x ,ufj 
v = (vi,v 2 ) in R 2 set 


(m,v) = 4m jVj + 5u 2 v 2 
Show that it defines an inner product. 


and 


Solution 

Certainly Axiom 1 is satisfied, because 

(u,vj= 4wiVi +5U2V2 = 4vi«i + 5 v2«2 = (v,m) . 

If tv = (w i, W 2 ), then 

(u + v,w) = 4(«j + Vj )w 1 +5 (n 2 + v 2 )w 2 

= 4w ,1V, + 5 m 2 iv 2 + 4v,iv, + 5 v 2 w 2 = + (v, w) 

This verifies Axiom 2. 

For Axiom 3, we have (cu,v) = 4(cMj)Vj +5(cm 2 )v 2 = c(4mjVj +5m 2 v 2 ) = c(u, v) 

For Axiom 4, note that (u,u) = 4mj 2 +5 u \ > 0, and 4mj 2 +5m 2 = 0 only if u\ = U2 = 0, that 
is, if u = 0. Also, (0,0)= 0. 
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So (u, v)= 4 mivi +5«2V2 = 4vi«i + 5v 2 U2 defines an inner product on R 2 . 


Example 2 

Let A be symmetric, positive definite n x n matrix and let u and v be vectors in 4?" 
Show that (u,v) =u , Av defines and inner product. 

Solution 

We check that 

(u,v) = u' Av = u.Av = Av.u 


Also 


And 


A'v.u = (v' A) u = v‘Au = (v,w) 


(u,v + vf) = u t A{y + w) = u t Av + u‘Aw 
= (u,vj + (u,w} 


(cu,vj = (cw) / Av = c(u r Av^ = c(u,v) 

Finally since A is positive definite 

(u,u\ = u'Au> 0 for all u ^ 0 

So (u,u) = u‘Au = 0 iff u = 0 
So {u,v)=u'Av is an inner product space. 


Example 3 

Let to, ..., t n be distinct real numbers. For p and q in P„, define 
{p,q) = p(t 0 )q(t 0 ) + p(tfq(tf + • • ■ + p(t n )q(t n ) 
Show that it defines inner product. 


Solution 

Certainly Axiom 1 is satisfied, because 

(p,q) = p(t 0 )q(t 0 ) + pffqff + • • ■ + p(t n )q(t n ) 

= g(t 0 )p(t 0 ) + qffpff + • • • + q(t„)p(t „) = (q, p) 

If r= r(t 0 ) + r(fj) + • • • + r(t n ), then 

{p + q,r) = [p(t 0 ) + q(t 0 )]r(t 0 ) + [p(t 1 ) + q(t 1 )]r(t l ) + --- + [p(t n ) + q(tj]r(t n ) 

= [pfo)ft 0 ) +Pf)r(tf + ... +p(tJr(tf\ + [q(tJr(t 0 ) + q(tfr(t l ) + ... + q(t n )r(tn)] 

= (p,r) + (q,r) 

This verifies Axiom 2. 

For Axiom 3, we have 

(cp, q) = [cp(t 0 )\ q(t 0 ) + [cp(tf\ q(tf + ■■■ + [cp(t n )\ q(t n ) 

= c [p(t 0 )q(t 0 ) + p(t 1 )q(t l ) + ■■■ + p(t„)q(t n )\ = c{p,q) 

For Axiom 4, note that 

(. P, P) = [P(t 0 )f +[p(t, )] 2 + • • • + [Pf, )] 2 > 0 
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Also, (0,0)=0. (We still use a boldface zero for the zero polynomial, the zero vector in 
P „.) If (p,p}= 0, then p must vanish at n + 1 points: to, ...,f n .This is possible only if/; is 
the zero polynomial, because the degree of p is less than n + 1. Thus 
(p,q) = p(t 0 )q(t 0 ) + -h p(t n )q(t n ) defines an inner product on P„. 

Example 4 

Compute (p,q) where p(t)= 4+t q(t) = 5-4t 2 
Refer to P 2 with the inner product given by evaluation at -1, 0 and 1 in example 2. 

Solution 

P(-1) = 3,P(0) = 4,P(1) = 5 
q(-l) = l,q(0) = 5,q(l) = l 

{p,q) = P(~ 1)<Z(-1) + ^( 0 )^( 0 ) + P(l)g(l) 

= (3)(l) + (4)(5) + (5)(l) 

= 3 + 20 + 5 


Example 5 

Compute the orthogonal projection of q onto the subspace spanned by p, for p and q in 
the above example. 

Solution 

The orthogonal projection of q onto the subspace spanned by p 

P(-1) = 3,P(0) = 4,P(1) = 5 
</(-!) = !, </(0) = 5, </(!) = ! 


q.p = 28 P-P = 50 


q.p 28.. . 

q =—P = ~ (4+0 

p.p 50 

56 14 

—- 1 - 1 

25 25 


Example 6 

Let V be P 2 , with the inner product from example 2 where 


t 0 = 0 , h = and t 2 = 1 


Let p(t^ = \2t 2 and q(t^ = 2t-\ 

Compute ( p,q ) and (q,q) 

Solution 
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(p,q)=p{b)q{ti) + p[^ + p(\)q(\) 
=(0)(-l) + (3)(0) + (l2)(l) =12 


(^)=[4°)T 


+ 


q 


( i A 




+ 


[HOT 


=(- l ) 2 +( 0) 2 + ( l) 2 =2 


Norm of a Vector 

Let V be and inner product space with the inner product 
denoted by (u,v) just as in R" , we define the length or norm of a vector V to be the 
scalar 

IMI = V( w,v ) 01 14= ( w,v ) 

1) A unit vector is one whose length is 1. 

2) The distance between u&v is ||w-v|| vectors u&v are orthogonal if ^u,v) = 0 


Example 7 


Compute the length of the vectors in example 3. 


Solution 


H 2 =(p»^) = W°)T 


+ 


f i A 


n2 




+ 


[HOT 


= (0) 2 +(3) 2 +(12) 2 =153 
| = Vl53 


In example 3 we found that 


{q,q)=2 

Hence ||g|| = V2 


Example 8 

Let tK 2 have the inner product of example 1 and let x=( 1,1) and y=(5,-l) 
a) Find ||x|| , |y|anj|(x,y}|" b) Describe all vectors (z,,z 2 ) that are 

orthogonal to y. 

Solution 


a) We have x=( 1,1) and y=(5,-l) 

And (x, y) = 4x,y, + 5 x 2 y 2 
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IHI = \K^x) = V 4 (1)(1) + 5(1)(1) 

= V4 + 5=V9 = 3 

| v|| = yj{yiy} = V 4(5)(5) + 5(-1)(-1) 
= VlOO + 5 = Vl05 


\(x,y)\-={x,y}{x,y) 

= [4(1)(5) + 5(1)(-1)] 2 
= [20 — 5] 2 
= [15] 2 =225 


b) All vectors z = (z 1 ,z 2 ) orthogonal to y=(5,-l) 

<y,z>= 0 


4(5)(z 1 ) + 5(-1)(z 2 ) = 0 
20zj -5z 2 =0 
4zj-z 2 =0 


[4 -1] 


= 0 


So all multiples of 


4 

-1 


are orthogonal to y. 


Example 9 

Le V be P A with the inner product in example 2 involving evaluation of 
polynomials at -2,-1,0,1,2 and view P 2 as a subspace of V. Produce an orthogonal basis 
for P 2 by applying the Gram Schmidt process to the polynomials 1 ,t&t 2 . 

Solution 


Given polynomials 1 


t 2 at -2,-1,0, 1 and 2 


Polynomial: 

1 

'll 


t 

~-2 


t 2 

~4~ 


1 


-1 


1 

Vector of values: 

1 


0 


0 


1 


1 


1 


1 


2 


4 


The inner product of two polynomials in V equals the (standard) inner product of their 
corresponding vectors in R 5 . Observe that t is orthogonal to the constant function 1. So 
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take po(t) = 1 and pi(t ) = t. For p 2 , use the vectors in R 5 to compute the projection of t 2 
onto Span {p 0 ,pi}: 

(t 2 ,p 0 ) = (t 2 , l) = 4 + 1 + 0 + 1 + 4 = 10 
(p 0 >Po) = 5 

(t 2 , Pl ) = {t 2 ,t) = - 8 + (-l) + 0 +1 + 8 = 0 


2 10 

The orthogonal projection of t onto Span {1, t} is —p 0 + Op v Thus 

p 2 (t) = t 2 -2p 0 (t) = f -2 
An orthogonal basis for the subspace P 2 of V is: 


Polynomial: p 0 pi 


P2 


Vector of values: 


'll 


~-2] 


' 2 ' 

1 


-1 


-1 

1 


0 


-2 

1 


1 


-1 

1 


2 


2 


Best Approximation in Inner Produce Spaces 

A common problem in applied mathematics involves a vector space V whose 
elements are functions. The problem is to approximate a function / in V by a function 
g from a specified subspace W of V. The “closeness” of the approximation of / 
depends on the way ||/-g|| is defined. We will consider only the case in which the 
distance between / and g is determined by an inner product. In this case the best 
approximation to / by functions in W is the orthogonal projection of / onto the 
subspace W. 


Example 10 

Let V be P 4 with the inner product in example 5 and let P 0 ,P l &P 2 
be the orthogonal basis for the subspace P 2 , find the best approximation to 


^ W = 5 -^ 4 


by polynomials in P 2 . 


Solution: 

The values of po, Pi, and p 2 at the numbers - 2, -1, 0, 1, and 2 are listed 

in R 5 vectors in 

Polynomial: p 0 pi p 2 
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'll 


'-2] 


' 2 ' 

1 


-1 


-1 

1 


0 


-2 

1 


1 


-1 

1 


2 


2 


Vector of values: 


The corresponding values for p are: -3,9/2,5,9/2, and -3. 

We compute 

{P’Po} = 8 (p,Pi) = 0 {p,p 2 ) = ~ 31 

{Po,Po) = 5 > {p 2 ,p 2 ) = M 

Then the best approximation in V top by polynomials in Pi is 

{P’Po) , (P,Pl) , {p,P 2 ) 

P = Proj Pi p = -f - V Po + 7- \ Pi + 7- \ P - 


{PvPo) ° (PvPi) 
= jPo+1£p 2 =f-ii(^ 2 -2). 


{pi’pi)' 


This polynomial is the closest to P of all polynomials in P 2 , when the distance between 
polynomials is measured only at -2, -1,0, 1, and 2. 

Cauchy - Schwarz Inquality 


For all u,v in V 
< // ||v|| 


Triangle Inequality 


Proof 


For all u,v in V 
II u + v|| < Hull + llvll 


M+V ={U + V,U + V) 

= (u,u} + 2(u,v') + (v,v} 
< ||n||" +2 |^m,v)| + ||v|| 2 
<||m||“ + 2 ||w|| 11 v 11 + ||v||- 

llu+v || 2 = (WuUWvt 


U+V = U +V 


Inner product for 
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Probably the most widely used inner product space for applications 
is the vector space C[a,h] of all continuous functions on an interval a < t < b , with an 
inner product that will describe. 

Example 11 

For /, g in C[a,h] , set 

(f,g)=\ a b f(t)g(t)dt 

Show that it defines an inner product on C[a,h]. 

Solution 

Inner product Axioms 1 to 3 follow from elementary properties of 

definite integrals 

l-(f,g) = (g,f) 

2 •{/ + h,g) = (f,g) + {h,g) 

3 -{cf,g) = c(f,g) 

For Axiom 4, observe that 

{f,f) = [[f(t)] 2 dt> 0 

2 

The function [/(/)] is continuous and nonnegative on [a, b\. If the definite integral of 
[f{t)\ is zero, then \f{t)\ must be identically zero on [a, b\, by a theorem in advanced 
calculus, in which case /is the zero function. Thus (/,/)= 0 implies that /is the zero 
function of [a, b]. 

So (f,g) = J f(t)g(t)dt defines an inner product on C[a, b\. 


Example 12 

Compute {fig) where f(t) = l-3t 2 and g(t)=t-t 3 on v = C[0,l]. 

Solution 

Let V be the space C[a,h] with the inner 
{f,g)=[ b f{t) g{t)dt 

fit) = l-3t 2 , g(t) = t-t 3 

{f,g) = \] ) {l-3t 2 )[t-f)dt 
= J o (3t 5 -4/ +t)dt 

1 6 4 1 o' 

=-r-r+-r 

2 2 

^ z. o 

= 0 


Example 13 


product 
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Let V be the space C[a,h] with the inner product 
{f ,g) = [ b f{t) g{t)dt 

Let W be the subspace spanned by the polynomials 

^(f) = i, p 2 (t) = 2t-i & p 3 (t) = nt 2 

Use the Gram - Schmidt process to find an orthogonal basis for W. 

Solution 


Let qi =pi, and compute 

{p 2 ,q,) = \l(2t-mdt = (t 2 -t)^ = 0 

So p 2 is already orthogonal to qi, and we can take q 2 = p 2 . For the projection of p 3 onto 
W 2 = Span {qi, q 2 }, we compute 

( p 3 ,qj) = £l2r -1 dt = 4 1 3 ^ 

= 1 


= 4 


Then 


qi,q 1 ) = \\-\dt = i 
p 3 ,q 2 ) = £ 12r (2 1 -1 )dt =£(24t 3 -12 t 2 )dt = 2 

(p 3 >4i) , {Ps’Vi} 4 ,2 „ , * 

P ro J»., Pi = 1 -f + 7-f = T ?/ + 777 = 4 qi + 6q 2 


And 
As a 

subspace tFis {qi,q 2 ,qi) 


<*>*> 1 1/3 

9i= Pi - proL Pj =P ; - 4?, - 6q 2 


As a function, = 12f 2 - 4 - 6(2t - 1) = 12f 2 - 12t + 2. The orthogonal basis for the 


Exercises 

Let tH 2 have the inner product of example 1 and let x=( 1,1) and y=(5,-l) 
a) Find ||x|| , ||y||am/ |(jc,_y)|" b) Describe all vectors (z,,z 2 ) that are 

orthogonal to y. 


2) Let S .H 2 have the inner product of Example 1. Show that the Cauchy-Shwarz 
inequality holds for x=(3,-2) and y=(-2,l) 


Exercise 3-8 refer to P 2 with the inner product given by evaluation at -1,0 and 1 in 
example 2. 

3) Compute (p,q) where p(t)= 4+t q(t) = 5-4t 2 

4) Compute (p,q) where p(t)= 3t -1 2 q(t) = 3 + t 2 

5) Compute \\P\\ and ||g|| for p and q in exercise 3. 

6) Compute \\P\\ and ||g| for p and q in exercise 4. 

7) Compute the orthogonal projection of q onto the subspace spanned by p, for p 
and q in Exercise 3. 
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8) Compute the orthogonal projection of q onto the subspace spanned by p, for p 
and q in Exercise 4. 

9) Let P 3 have the inner product given by evaluation at -3,-1,1, and 3. Let 
P 0 (t) = 1 , Pi(t) = t, and p 2 (t) = t 2 

ajComputer the orthogonal projection of P 2 on to the subspace spanned by P 0 and P l . b) 
Find a polynomial q that is orthogonal to P 0 and P x such tha {p 0 ,p x ,q} is an orthogonal 
basis for span {p 0 ,p x ,q] . Scale the polynomial q so that its vector of values at (-3,- 
1,1,3) is (1,-1,-1,1) 

10) Let P 3 have the inner product given by evaluation at -3,-1,1, and 3. Let 
Po(t) = 1 , Pi(t) = t, and p 2 (t) = r 

Find the best approximation to pit ) = t 3 by polynomials in Span [p 0 ,p x ,q] ■ 

11 ) Let p 0 ,p x ,p 2 be the orthogonal polynomials described in example 5, where the 
inner product on P 4 is given by evaluation at -2, -1, 0, 1, and 2. Find the orthogonal 
projection of t 3 onto Span {p 0 ,p 1 ,p 2 \ 

12) Compute (/,g) where f(t) = l-3t 2 and g(t) = t-t 3 on v = C[0,l]. 

13) Compute //,g\ where 

f(t) = 5t-3 and g(t) = t 3 -t 2 on v = C [0, l]. 

14) Compute ||/|| for f in exercise 12. 

15) Compute ||g|| for g in exercise 13. 


16) Let V be the space C[-2,2] with the inner product of Example 7. Find an 
orthogonal basis for the subspace spanned by the polynomials l,t,t 2 . 


17) 



u. 


V, 

Let u = 

l 

_u 2 '_ 

and v = 

1 

3_ 


defines an inner product. 


be two vectors in R 2 . Show that (m,v) 


2mjVj + 3w 2 v 2 
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Lecture 44 

Application of inner product spaces 


Definition 

An inner product on a vector space V is a function that associates to each pair of vectors u 
and v in V, a real number (u,v) and satisfies the following axioms, for all u, v, tv in V 

and all scalars c: 

1. (u,v) = (v,u) 

2. (u + v, = (u, w'j + ^v, w^j 

3. (cu,v} = c(u,v) 

4. {u,ii)>0 and (u,u} = 0 iff u = 0. 

A vector space with an inner product is called an inner product space. 

Least Squares Lines 

The simplest relation between two variables x and y is the linear 
equation y = /3 0 + J3 x x . Often experimental data produces points {x x ,y x ),...,(x n ,y n ) that 
when graphed, seem to lie close to a line. Actually we want to determine the parameters 
1 3 0 and /?, that make the line as “close” to the points as possible. There are several ways 
to measure how close the line is to the data. The usual choice is to add the squares of the 
residuals. The least squares line is the line y = J3 0 + that minimizes the sum of the 
squares of the residuals. 

If the data points are on the line, the parameters /3 0 and /?, would satisfy the 
equations 

predicted Observed 
value value 


A,+M =T, 

Ao+A*2 =t 2 


Ao+M, =y„ 
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We can write this system as 

xp=y 



1 x, 


y 0 


1 x 2 




Fi 

Where X = 


, P = 

Po 

, y = 





P _ 




1 X n\ 


y n _ 


Computing the least-squares solution of X /3 =y is equivalent to finding the /3 that 
determines the least-squares line. 

Example 1 

Find the equation y = J3 0 + f\x of the least-squares line that best fits the data points 
(2, 1), (5, 2), (7, 3), (8, 3). 


Solution 


xp=y 


"i 

2] 




V 

i 

i 

5 

, P = 

~Po] 

» y = 

2 

7 

A J 

3 

i 

8_ 




_3_ 


Here X = 


For the least-squares solution of xft = y , obtain the normal equations(with the new 
notation) : 

X T Xj3 = X T y 
i.e, compute 


X‘X = 


X T y = 



"1 2" 



"ii i r 

1 5 


1 

<N 

<N 

i_ 

2 5 7 8_ 

1 7 


22 142 


! 8_ 




"1 1 

1 

r 

V 

2 


"9 ' 

_2 5 

7 

8_ 

3 


57 




_3_ 
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The normal equations are 


“4 

22“ 

A* 


“ 9“ 

22 

142 

A. 


_57_ 


Hence, 


A* 


A. 



4 

22“ 

-1 

“ 9“ 

22 

142 


57 


1 

“142 

-22“ 

“ 9“ 

84 

-22 

4 

57 


1 

84 


24 

30 


2/7 ' 
5/14 


Thus, the least -squares line has the equation 

2 5 

y = — H- x 

' 7 14 



0 


0 


5 

x 


Weighted Least-Squares 

Let y be a vector of n observations, y 1 ,y 2 ,...,y„ and suppose we wish to 

approximate y by a vector y that belongs to some specified subspace of M" (as discussed 
previously that y is written as Ax so that y was in the column space of A).Now suppose 
the approximating vector y is to be constructed from the columns of matrix A. Then we 
find an x that makes Ax = y as close to y as possible. So that measure of closeness is the 
weighted error 

\Wy-Wyf =\\Wy-WAy\f 
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Where W is the diagonal matrix with (positive) w 1 ,...,w n on its diagonal, that is 
w i 0 ... 0 

0 w 2 




L° • • • w «\ 

Thus, x is the ordinary least-squares solution of the equation 

WAx = Wy 


The normal equation for the weighted least-squares solution is 
(WAf WAX = (WA) r Wy 

Example 2 


Find the least squares line v = /i (l + P x x that best fits the data (-2, 3), (-1, 5), (0, 5), (1, 4), 

(2, 3). Suppose that the errors in measuring the v-values of the last two data points are 
greater than for the other points. Weight this data half as much as the rest of the data. 

Solution 

Write X, (3 and y 


"1 

-2] 


"3" 

1 

-1 


5 

1 

0 

,P = 

Po\ 

, y = 

5 

1 

1 


A1 


4 

1 

2 


3 


For a weighting matrix, choose W with diagonal entries 2,2,2 , 1 and 1. 
Left-multiplication by W scales the rows of X and y: 


WX = 


"2 

0 

0 

0 

0 " 

"1 - 2 ~ 

0 

2 

0 

0 

0 

1 -1 

0 

0 

2 

0 

0 

1 0 

0 

0 

0 

1 

0 

1 1 

0 

0 

0 

0 

1 

1 2 
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1 

1 

<N 
l_ 


"6 ~ 

2 -2 


10 

2 0 

,Wy = 

10 

1 1 


4 

1 2 


3 


WX = 


For normal equation, compute 


(WXfWX 
And solve 


14 

-9 


14 

-9 


-9 

25 


and ( WX) T Wy = 


59 

-34 


- 

9 ~ 

Ao 1 


' 59" 



25_ 

Ail 


-34 



A 0 ] 


"14 


-9 

-1 

' 59" 

A. 


-9 


25 _ 


_-34_ 


Ao' 

l 

"25 

9" 

' 59' 

Ail 

“269 

9 

14 

_-34_ 


Ao 1 

l 

"25 

9" 

' 59" 

Ail 

“269 

9 

14 

_-34_ 


Ao 

l 

"1169" 


"4.3" 

Ai _ 

“ 269 

55 


0.2 


Therefore, the solution to two significant digits is J3 0 = 4.3 and /?, = 0.20 . 
Hence the required line is y = 4.3 + 0.2x 

In contrast, the ordinary least-squares line for this data can be found as: 




"1 -2] 






1 -1 





"l 1111" 



"5 

0" 

X r X = 

-2-10 1 2 

1 0 

— 

0 

10 



1 1 






1 2 
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3 

5 


"1 1111 

5 

_ 

"20" 

-2-10 1 2 



-1 


4 


— 


3 




“5 

O' 

"A>1 


" 20" 

0 

10 

A! 


-1 


"A 0 1 


'5 

O' 

-1 

" 20" 

A\ 


0 

10 


-1 


"A," 

i 

"10 

0" 

r 20' 

A. 

“ 50 

0 

5 


"AT 

1 

' 200" 


4.0" 

A. 

“ 50 

_ -5 _ 


- 0 .! 


Hence the equation of least-squares line is 
v = 1.0-0. lx 


> 



FIGURE 1 Weighted and 
ordinary least-squares lines. 


What Does Trend Analysis Mean? 

An aspect of technical analysis that tries to predict the future movement of a stock based 
on past data. Trend analysis is based on the idea that what has happened in the past gives 
traders an idea of what will happen in the future. 
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Linear Trend 


A first step in analyzing a time series, to determine whether a linear relationship provides 
a good approximation to the long-tenn movement of the series computed by the method 
of semi averages or by the method of least squares. 

Note 


The simplest and most common use of trend analysis occurs when the points t 0 ,t v ...,t n 
can be adjusted so that they are evenly spaced and sum to zero. 

Example 

Fit a quadratic trend function to the data (-2,3), (-1,5), (0,5), (1,4) and (2,3) 

Solution 


The t-coordinates are suitably scaled to use the orthogonal polynomials found in Example 
5 of the last lecture. We have 


Polynomial: 

Po 

Pi 

P 2 

data 

:g 


T 


“-2“ 


“ 2“ 


“3" 


1 


-1 


-1 


5 

Vector of values : 

1 


0 


-2 


5 


1 


1 


-1 


4 


1 


2 


2 


3 


<P 0 ’Po> <PvPi> < Pi 5 Pi > 





and p(t) = 4-0.1t-0.5(t 2 -2) 

Since, the coefficient of p 2 is not extremely small, it would be reasonable to conclude 
that the trend is at least quadratic. 
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Above figure shows that approximation by a quadratic trend function 

Fourier series 

If f is a 2 n -periodic function then 

fit) = ~+ X C0S mt + b m Sil1 mt ) 

^ m=\ 

is called Fourier series of f where 

1 2 n 

a „, = — [ fit) cos mt dt and 
n o 

Y 2-7T 

b m = — [ /(t) sin mt dt 
* o 


Example 

Let C[0,2/r] has the inner product 

2 71 

</,&>={ fifgifdt 
o 

and let m and n be unequal positive integers. Show that cos mt and cos nt 
are orthogonal. 

Solution 

When m ^ n 


2 K 

< cos mt, cos nt >= J cos mt cos nt dt 

o 


2 


2k 

J [cos + nt) + cos dt 

o 
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sin (mt + nt) 
m + n 


sin (mt - nt) 


m - n 


2n 


Example 

Find the nth-order Fourier approximation to the function 
f(t) = t on the interval [0,2^-]. 

Solution 

We compute 


In 1 

\ tdt = — 

1 2 
-r 

2 n 

J 2 7T 

0 

2 

0 


and for k>0, using integration by parts, 


1 2 r , , 1 

= — \t cos kt at = — 
n { n 


■Xr cos kt +—sin kt 
k 2 k 


2 n 


= 0 


b k = — [ tsmktdt = — 
n { n 


\-smkt- — cos kt 
k 2 k 


2 n 


Thus, the nth-order Fourier approximation of f(t) = t is 


2 

k 


n - 2 sin t - sin 2t — sin 3?-sin nt 

3 n 

The nonn of the difference between f and a Fourier approximation is called the mean 
square error in the approximation. 

It is common to write 


fit) = ^~ + Y J (a m cos mt + b m sin mt) 

^ m= 1 


This expression for f (t) is called the Fourier series for f on[0,2^]. The term a m cos mt , 
for example, is the projection of f onto the one-dimensional subspace spanned by cos mt. 
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Example 

Let q x (t ) = 1, q 2 (t ) = t,andq 3 (t) = 3 1 2 -4 .Verify that {q ] ,q 2 ,q 3 } is an orthogonal set in 
C {-2,2] with the inner product 

b 

<f,g>=\f(t)g(t)dt 


Solution: 

2 

< q x , q 2 >= J 1 .t dt 

-2 



= 0 


< q x , q 3 >= 11 .(3 1 2 - 4) dt = ( t 2 - 4t)| ^ = 0 

-2 


2 

< ^ 2?^3 >= J ^-( 3^ 2 

-2 


_4 )dt=(-t 4 
4 



= 0 


1-2 


Exercise 


1. Find the equation y = J3 0 + fd x x of the least-squares line that best fits the data 
points (0, 1), (1, 1), (2, 2), (3,2). 

2. Find the equation y = J3 0 + fd x x of the least-squares line that best fits the data 
points (-1,0), (0, 1), (1, 2,),(2, 4). 

3. Find the least-squares line y = /?„ + (\x that best fits the data 

(-2, 0), (-1, 0), (0, 2,),(1, 4),(2, 4), assuming that the first and last data points are 
less reliable. Weight them half as much as the three interior points. 

4: To make a trend analysis of six evenly spaced data points, one can use orthogonal 
polynomials with respect to evaluation at the points t=-5, -3,-1, 1, 3and 5 

(a) . Show that the first three orthogonal polynomials are 

3 2 35 

Po(t) = \, Pi(t) = t, and p 2 (t) = -t~- — 

O O 

(b) Fit a quadratic trend function to the data 
(-5, 1), (-3, 1), (-1,4), (1,4), (3,6), (5,8) 

5: For the space C[0, 2k] with the inner product defined by 

2 7T 

<f,g>= J fif) gif) dt 

o 

(a) Show that sin mt and sin nt are orthogonal when m A n 

(b) Find the third-order Fourier approximation to f{t) = 2k - 1 

(c) Find the third order Fourier approximation to cos 3 1 , without performing any 
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integration calculations. 

6: Find the first-order and third order Fourier approximations to 
f(t) = 3-2sint + 5sin2t-6cos2t 
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Revision of Previous Lectures 
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